ProTips

Qore Development ProTips

Performance and Memory

xrange() vs range()

Motivation: Performance

xrange() should generally by used instead of range() for iterating iterators since it's much more efficient; range() returns a real list of integers, while xrange() returns an iterator that programmatically iterates a range of integers (and therefore can iterate arbitrarily-large integer ranges with a constant memory footprint).

map vs foreach

Motivation: Performance

map should be used instead of foreach where performance is critical and where possible (when a single expression is iterated instead of one or more statements); there is an overhead to executing statements, and the map operator makes up a single expression, and is therefore much faster and much more efficient in most cases than foreach. Note that in cases where clarity is more important than performance, the foreach statement can provide easier to understand code

Regular Expressions

Motivation: Performance

Use regular expression operators instead of regular expression functions when the regular expression arguments are known at development time. This allows the regular expressions to be parsed once at parse time (O(1)) instead of every time at runtime (O(n)).

faster:

$item.description =~ s/[^[:ascii:]]/ /g;

slower:

$item.description = regex_subst($item.description, '[^[:ascii:]]', ' ', Qore::RE_Global);

Additionally the first variant above can be executed in-place, whereas the second variant requires the string to be copied, the regular expression to be applied to the copy, then for the original string to be destroyed and replaced with the copy (in addition to the run-time compilation of the regular expression every time it's executed).

Use keys <hash> to Iterate Hash Keys

Motivation: Performance and Memory

By using the keys operator instead of <hash>::keys() or <hash>::keyIterator() to iterate hash keys, the hash is efficiently iterated in place using lazy evaluation for functional and list operators and iterator statements rather than creating a single object that iterates the hash or creating a dynamic list consisting of string values corresponding to each hash key.

more efficient:

# iterates the hash's keys in place using lazy functional evaluation
string str = foldl $1 + ", " + $2, keys my_hash;

foreach string key in (keys my_hash)
    check(key);

map check($1), keys my_hash;

less efficient:

# creates a single object that iterates through the hash's keys in place
map check($1), my_hash.keyIterator();

# dynamically creates a list of strings of the hash's keys at runtime and iterates the resulting list
foreach string key in (my_hash.keys())
    check(key);

map check($1), my_hash.keys();

Note: in Qore pre-0.8.13, <hash>::keyIterator() is preferred over the keys operator because lazy functional evaluation in the keys operator was introduced in Qore 0.8.13

Creating Large Hashes

Motivation: Performance

It's better to create a large hash in one statement instead of in n statements since it executes much faster (ie requires less CPU time), as in the following examples:

faster:

hxml.ExpectedDelivery = (
    "XMLSchemaVersion"     : 1,
    "ID"                   : %order_id,
    "CreationDate"         : format_date('YYYY-MM-DDTHH:mm:SSZ', %status_date),
    "OracleDeliveryNumber" : %order_id,
    "OracleOrderNumber"    : %orig_order_id,
    "CustomerPONumber"     : %customer_po_number,
    "Notes"                : %delivery_text,
    "UserID"               : 'QORUS',
    "Location"             : 'BFP',
    "SourceID"             : 1,
    "SourceType"           : 5,
    "ExpectedDeliveryItem" : (),
);

slower::

hxml.ExpectedDelivery.XMLSchemaVersion  = 1;
hxml.ExpectedDelivery.ID                = %order_id;
hxml.ExpectedDelivery.CreationDate      = format_date('YYYY-MM-DDTHH:mm:SSZ', %status_date);
hxml.ExpectedDelivery.OracleDeliveryNumber = %order_id;
hxml.ExpectedDelivery.OracleOrderNumber = %orig_order_id;
hxml.ExpectedDelivery.CustomerPONumber  = %customer_po_number;
hxml.ExpectedDelivery.Notes             = %delivery_text;
hxml.ExpectedDelivery.UserID           = 'QORUS';
hxml.ExpectedDelivery.Location         = 'BFP';
hxml.ExpectedDelivery.SourceID         = 1;
hxml.ExpectedDelivery.SourceType       = 5;
hxml.ExpectedDelivery.ExpectedDeliveryItem = ();

Avoid inlist() and Never Use inlist() With Hash Keys

Motivation: Performance and Memory

inlist() is O(n) and therefore should never should be used with hash keys; hash key lookups are O(ln(n)) in the worst case and O(1) in the best case. Use hash::hasKey() instead.

correct:

if (hash.hasKey(key))
    doit();

incorrect:

if (inlist(key, hash.keys()))
    doit();

Value Type Testing

Motivation: Performance

Use <any>::typeCode() instead of <any>::type() to make type comparisons because integer operations are much faster and more efficient than string operations.

faster:

if (value.typeCode() == NT_BOOLEAN)
     doSomething();

slower:

if (value.type() == Type::Boolean)
    doSomething();

Exception-Safe Programming

on_exit, on_success, on_error vs RAII

Motivation: Code Quality

Qore supports D-style lexical-scope related resource management and exception-safe programming with the on_exit, on_success, and on_error statements, as well as RAII

The use of both techniques are equally valid; consider the following examples:

Mutex m();
{
    m.lock();
    on_exit m.unlock();
}

and (the RAII example):

Mutex m();
{
    AutoLock al(m);
}

Both of the above are clear and implement exception-safe resource management of the Mutex lock. The first example may be slightly more efficient than the second due to the overhead of the AutoLock object creation.

Miscellaneous

Make Scripts Executable

Motivation: Convenience

Always start your scripts with an appropriate hash-bang and set the executable bits as in the following examples:

#!/usr/bin/env qore

This saves the user from having to type in the name of the executable before the script name every time the script is executed.

Coding Style

Concise Object Construction

Motivation: Style

The concise form of object construction in the variable's declaration; ex:

more efficient:

Mutex m();

is preferred over the long style; ex:

less efficient:

Mutex m = new Mutex();

This is mostly to reduce typing overhead.

Command-Line Argument Processing

Motivation: Code Maintainability

The GetOpt class should be used; note that when defining options, the short option should always be defined first in the string to make it easier to see the short options when adding options and to detect duplicates.

correct:

const Opts = (
    "url"   : "u,url=s",
    "proxy" : "p,proxy-url=s",
    "fgn"   : "f,foreign",
    "xml"   : "x,xml",
    "lxml"  : "X,unformatted-xml",
    "verb"  : "v,verbose",
    "lit"   : "l,literal:i+",
    "show"  : "W,show-url",
    "to"    : "t,timeout=i",
    "mon"   : "m,monitor",
    "remote": "Q,remote=s",
    "help"  : "h,help",
);

In the above example, the short options are easily identifiable since the are defined first in the option string.

incorrect:

const Opts = (
    "url"   : "url,u=s",
    "proxy" : "proxy-url,p=s",
    "fgn"   : "foreign,f",
    "xml"   : "xml,x",
    "lxml"  : "unformatted-xml,X",
    "verb"  : "verbose,v",
    "lit"   : "literal,l:i+",
    "show"  : "show-url,W",
    "to"    : "timeout,t=i",
    "mon"   : "monitor,m",
    "remote": "remote,Q=s",
    "help"  : "help,h",
);

In the above example, the short options are not easily identifiable; with a large option hash, it would be more difficult to find the short options and to recognize duplicates.

Use of Complex Types

Motivation: Performance and Maintainability

It is recommended to use complex types when possible to ensure maximum performance and maintainability.

recommended:

# a common way to emulate a set is to assign True to a set of hash keys
hash<string, bool> my_set = {
    "value1": True,
    "value2": True,
};

hash<auto> sub get_some_hash() {
    return {
    };
}

hash<auto> my_hash = get_some_hash();

list<string> my_keys = keys my_set;

not recommended:

hash my_set = {
    "value1": True,
    "value2": True,
};

hash sub get_some_hash() {
    return {
    };
}

hash my_hash = get_some_hash();

list my_keys = keys my_set;

Using complex types improves maintainability and makes APIs more useful as the types used are clear and unambiguous.

Furthermore there is a performance improvement as declaring lvalues as hash or list strips any complex types from any value passed to it at runtime for backwards compatibility.

This type stripping was implemented as a compromise to allow Qore to introduce complex types ubiquitously. If Qore would not strip the complex types from hash and list declarations, it would result in widespread breakage with existing code, which would have limited the usefulness of complex types. By implementing type stripping for backwards compatibility, Qore was able to deploy complex types in many places in builtin and system APIs.

Therefore due to type stripping, hash<auto> is preferred over hash and list<auto> and softlist<auto> are preferred over list and softlist.