OmniThreadLibrary is a multithreading library for Delphi, written mostly by the author of this book (see Credits for full list of contributors). OmniThreadLibrary can be roughly divided into three parts. Firstly, there are building blocks that can be used either with the OmniThreadLibrary threading helpers or with any other threading approach (f.i. with Delphi’s TThread or with AsyncCalls). Most of these building blocks are described in chapter Miscellaneous, while some parts are covered elsewhere in the book (Lock-free Collections, Blocking collection, Synchronization).
Secondly, OmniThreadLibrary brings low-level multithreading framework, which can be thought of as a scaffolding that wraps the TThread class. This framework simplifies passing messages to and from the background threads, starting background tasks, using thread pools and more.
Thirdly, OmniThreadLibrary introduces high-level multithreading concept. High-level framework contains multiple pre-packaged solutions (so-called abstractions; f.i. parallel for, pipeline, fork/join …) which can be used in your code. The idea is that the user should just choose appropriate abstraction and write the worker code, while the OmniThreadLibrary provides the framework that implements the tricky multithreaded parts, takes care of synchronisation and so on.
OmniThreadLibrary requires at least Delphi 2007 and doesn’t work with FreePascal. The reason for this is that most parts of OmniThreadLibrary use language constructs that are not yet supported by the FreePascal compiler.
High-level multithreading framework requires at least Delphi 2009.
OmniThreadLibrary currently only targets Windows installation. Both 32-bit and 64-bit platform are supported.
OmniThreadLibrary is an open-sourced library with the OpenBSD license.
This software is distributed under the BSD license.
Copyright (c) 2017, Primoz Gabrijelcic
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
- The name of the Primoz Gabrijelcic may not be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
In short, this means that:
In case your company would like to get support contract for the OmniThreadLibrary, please contact me.
usesstatement and start using the library!
OmniThreadLibrary includes one design-time component (
TOmniEventMonitor) which may be used to receive messages sent from the background tasks and to monitor thread creation/destruction. It is used in some of the demo applications.
To compile and install the package containing this component, follow these steps:
TOmniEventMonitorcomponent was installed.
You should repeat these steps whenever the OmniThreadLibrary installation is updated.
OmniThreadLibrary approaches the threading problem from a different perspective than TThread. While the Delphi’s native approach is oriented towards creating and managing threads on a very low level, the main design guideline behind OmniThreadLibrary is: “Enable the programmer to work with threads in as fluent way as possible.” The code should ideally relieve you from all burdens commonly associated with multithreading.
OmniThreadLibrary was designed to become a “VCL for multithreading” – a library that will make typical multithreading tasks really simple but still allow you to dig deeper and mess with the multithreading code at the operating system level. While still allowing this low-level tinkering, OmniThreadLibrary allows you to work on a higher level of abstraction most of the time.
There are two important points of distinction between TThread and OmniThreadLibrary, both explained further in this chapter. One is that OmniThreadLibrary focuses on tasks, not threads and another is that in OmniThreadLibrary messaging tries to replace locking whenever possible.
By moving most of the critical multithreaded code into reusable components (classes and high-level abstractions), OmniThreadLibrary allows you to write better multithreaded code faster.
In OmniThreadLibrary you don’t create threads but tasks. A task can be executed in a new thread or in an existing thread, taken from the thread pool.
A task is created using
CreateTask function, which takes as a parameter a global procedure, a method, an instance of a
TOmniWorker class (or, usually, a descendant of that class) or an anonymous method (in Delphi 2009 and newer).
CreateTask returns an
IOmniTaskControl interface, which can be used to control the task. A task is always created in suspended state and you have to call
Run to activate it (or
Schedule to run it in a thread pool).
The task has access to the
IOmniTask interface and can use it to communicate with the owner (the part of the program that started the task). Both interfaces are explained in full detail in chapter Low-level multithreading.
The distinction between the task and the thread can be summarized in few simple words.
Task is part of code that has to be executed.
Thread is the execution environment.
You take care of the task, OmniThreadLibrary takes care of the thread.
I believe that locking is evil. It leads to slow code and deadlocks and is one of the main reasons for almost-working multithreaded code (especially when you use shared data and forget to lock it up). Because of that, OmniThreadLibrary tries to move as much away from the shared data approach as possible. Cooperation between threads is rather achieved with messaging.
If we compare shared data approach with messaging, both have good and bad sides. On the good side, shared data approach is fast because it doesn’t move data around and is less memory intensive as the data is kept only in one copy. On the bad side, locking must be used to access data which leads to bad scaling (slowdowns when many threads are accessing the data), deadlocks and livelocks.
The situation is almost reversed for the messaging. There’s no shared data so no locking, which makes the program faster, more scalable and less prone to fall in the deadlocking trap. (Livelocking is still possible, though.) On the bad side, it uses more memory, requires copying data around (which may be a problem if shared data is large) and may lead to complicated and hard to understand algorithms.
OmniThreadLibrary uses custom lock-free structures to transfer data between the task and its owner (or directly between two tasks). The system is tuned for high data rates and can transfer more than million messages per second. However, in some situations shared data approach is necessary and that’s why OmniThreadLibrary adds significant support for synchronisation.
Lock-free (or microlocked) structures in OmniThreadLibrary encompass:
OmniThreadLibrary automatically inserts two bounded queues between the task owner (
IOmniTaskControl) and the task (
IOmniTask) so that the messages can flow in both directions.
Due to implementation details, OmniThreadLibrary requires that each thread owner maintains and processes a message queue. This condition is automatically satisfied in VCL and service applications, but running OmniThreadLibrary threads from a console application requires more work. Additional work is also required if you are creating OmniThreadLibrary threads from background threads.
To correctly use OmniThreadLibrary in a console application, said application
must process Windows messages. This is demonstrated in the
which is reproduced below.
The main program creates a future which runs a function
and then waits for it to return a value (
while not calc.IsDone).
The future waits five seconds and each second sends a message back to the owner
task.Comm.Send(MSG_STATUS, '... still calculating')). Main thread processes
these messages in the
If you run the program, you’ll see that the message
... still calculating
is displayed five times with one second delay between two messages. After that,
And the answer is: 42 is displayed.
The critical part of this program are two lines:
If you comment them out, the program will write
Background thread is calculating ...,
then nothing will happen (visibly) for five seconds and then all messages will be
displayed at once.
In this example it is not really critical to process messages as the program would continue to function correctly – more or less – even when message processing is removed, but in other cases all kinds of weird behaviour can occur. OmniThreadLibrary occasionally uses messages for internal purpose and if you prevent processing of these messages, applications may misbehave. The best approach is to always include periodic calls to ProcessMessages in a console application.
Similar considerations take order when an OmniThreadLibrary task is started from another OmniThreadLibrary task – the ‘intermediate’ thread (the one controlled by the task that creates another task) must process messages. The easiest way to achieve that is by using the MsgWait qualifier when creating a task.
Let’s take a look at an example from the
In this example, a click on a button creates a task that will create a future.
The critical part here is the call to
MsgWait which causes internal loop in
the task to process Windows messages. Without this
MsgWait the program would
The worker does all the work in its
This code executes from the context of a background worker thread.
It looks complicated, but actually it just creates a Future calculation (
FCalc := Parallel.Future<integer>)
and sets up event handlers that will process messages sent from the future
.OnMessage) and handle the completion of the future calculation (
OnMessage just takes message that was sent from the future, adds some text and current thread ID and
forwards message to the form where it is logged in the
WMLog method (not shown here).
OnTerminated also logs the event, clears the future interface and terminates
Task.Terminate). After that, form’s
TaskTerminated method (not shown here)
is called and cleans the task controller interface.
The future itself does nothing special, it just sends five messages with one second delay between them and then returns a value.
When you run the program and click on the button, following text will be displayed (thread IDs – numbers in brackets – will be different in your case, of course).
We can see that Future messages were generated in thread 18420, then passed through the parent thread 6088 and ended in the main thread 11968.
We have to do more work if we want to create a OmniThreadLibrary thread from
a basic Delphi
TThread thread. We have to make sure that thread messages are
periodically process by calling the
DSiProcessThreadMessages function from
DSiWin32 unit (or a similar code that calls
66_ThreadsInThreads demo contains an example of such situation.
We don’t have to do anything special when creating a thread.
The main thread method firstly creates a Future and sets
.OnMessage handler which just resends messages to the main thread.
Then it enters a
repeat .. until loop in which it waits for a Windows message
MsgWaitForMultipleObjects), processes all waiting messages (
and checks whether the calculation has completed (
At the end it cleans up the future and exits. That destroys the
Calling MsgWaitForMultipleObjects is not strictly necessary. You could just
DSiProcessThreadMessages from time to time. It does, however, improve the
performance as the code uses no CPU in such wait.
If you are already using some other kind of wait-and-dispach mechanism in your thread
then they are easy to convert to
Running the program shows similar behaviour as in the previous example.
TOmniValue (part of the OtlCommon unit) is data type which is central to the whole OmniThreadLibrary.
It is used in all parts of the code (for example in a communication subsystem) when type of the data that is to be stored/passed around is not known in advance.
In all cases ownership of reference-counted data types (strings, interfaces) is managed correctly so no memory leaks can occur when such type is stored in a
TOmniValue type is too large to be shown in one piece so I’ll show various parts of its interface throughout this chapter.
The content of a
TOmniValue record can be accessed in many ways, the simplest (and in most cases the most useful) being through the
While the setters for those properties are pretty straightforward, getters all have a special logic built in which tries to convert data from any reasonable source type to the requested type. If that cannot be done, an exception is raised.
For example, getter for the
AsString property is called
CastToString and in turn calls
TryCastToString, which is in turn a public function of
When you don’t know the data type stored in a
TOmniValue variable and you don’t want to raise an exception if compatible data is not available, you can use the
TryCastToXXX family of functions directly.
Alternatively, you can use
CastToXXXDef functions which return a default value if current value of the
TOmniValue cannot be converted into required data type.
They are all implemented in the same value, similar to the
For situations where you would like to determine the type of data stored inside
TOmniValue, there is the
IsXXX family of functions.
Alternatively, you can use the
There are two ways to clear a
TOmniValue – you can either call its
method, or you can assign to it a
TOmniValue also implements several
Implicit operators which help with
automatic conversion to and from different data types. Internally, they are
implemented as assignment to/from the
Implicit conversion to/from
TDateTime is supported only in Delphi XE and newer.
Equal operators simplify comparing
TOmniValue to an
Few methods simplify using
TOmniValue with a generic class/record.
CastFrom<T> converts any type into a
TOmniValue. In Delphi 2009, this function is severely limited as only very simple types (integer, object) are supported. In Delphi 2010 and newer,
TValue type is used to facilitate the conversion and all data types supported by the
TOmniValue can be converted.
TOmniValue into any other type. In Delphi 2009 same
limitations apply as for
CastToObject<T> (available in Delphi 2010 and newer) performs a hard cast with no type checking. It is equivalent to using
ToObject<T> (available in Delphi 2010 and newer) casts the object to type
with type checking. It is equivalent to using
AsObject as T.
Wrap<T> [3.06] wraps any data type in an instance of TOmniRecordWrapper<T>
and stores this value in a
Unwrap<T> [3.06] unwraps a
TOmniValue holding a TOmniRecordWrapper<T>
and returns owned value of type
T. It has to be called in this form:
Unwrap are especially useful as they allow you to store
TMethod data (event handlers) in a
TOmniValue can contain an array of other
they are stored in a TOmniValueContainer object. This object can be accessed directly by reading the
IsArray can be used to test whether a
TOmniValue contains an array of values.
Arrays can be accessed by an integer indexes (starting with
0), or by string
indexes (named access). Integer-indexed arrays are created by calling
TOmniValue.Create and string-indexed arrays are created by calling
In the latter case, elements of the
values parameter must alternate between names (string indexes) and values.
In the example above, both
ov['Key1'] would return the same
'Value of ov[''Key1'']'.
Array elements can be accessed with the
AsArrayItem property, by using
an integer index (for integer-indexed arrays), a string index (for
string-indexed arrays), or a
TOmniValue index. In the last case, the type of
data stored inside the
TOmniValue index parameter will determine how the array
element is accessed. This last form is not available in Delphi 2007, where
AsArrayItemOV should be used instead.
All forms of
AsArrayItem allow extending an array. If you write data into
an index which doesn’t already exist, the array will automatically grow to
accomodate the new value.
If you want to test whether an array element exists, use the
In Delphi 2010 and newer
TOmniValue also implements functions for converting
data to and from
TArray<T> for any supported type.
CastTo<T> functions are used internally to do the
T can be stored inside a
TOmniValue by calling the
function. To extract the data back into a record, use the
TOmniValue jumps through quite some hoops to store a record. It is first
converted into a TOmniRecordWrapper which is then
wrapped inside an IOmniAutoDestroyObject
interface to provide a reference-counted lifetime management.
Because of all that storing records inside
TOmniValue is, although very flexible, not that fast.
TOmniValue can take an ownership of a
TObject-type data. To achieve that,
you can either assign an object to the
AsOwnedObject property or set the
OwnsObject property to
When a object-owning
TOmniValue goes out of scope, the owned object
is automatically destroyed.
You can change the ownership status at any time by assigning to the
In Delphi 2010 and newer,
TOmniValue also provides a
Implicit operator so you can easily convert a
TOmniValue and back.
For people with special needs (and for internal OmniThreadLibrary use),
TOmniValue exposes following public methods.
_AddRef increments reference counter of stored data if
TOmniValue contains such data.
_Release decrements reference counter of stored data if
TOmniValue contains such data.
_ReleaseAndClear is just a shorthand for calling a
_Release followed by a call to
RawData returns pointer to the data stored in the
RawZero clears the stored data without decrementing the reference counter.
The OtlCommon unit also implements a simple object which can wrap a
TOmniValue for situations where you would like to store it inside a data structure that only supports object types.
OmniThreadLibrary heavily uses fluent interface approach. Most of functions in OmniThreadLibrary interfaces are returning
Self as the output value. Take for example this declaration of the pipeline abstraction, slightly edited for brevity.
As you can see, most of the functions return the
IOmniPipeline interface. In code, this is implemented by returning
This allows calls to such interfaces to be chained. For example, the following code from the Pipeline section shows how to use
Parallel.Pipeline without ever storing the resulting interface in a variable.
If you don’t like fluent interface approach, don’t worry. OmniThreadLibrary can be used without it. You can always call a function as if it is a procedure and compiler will just throw away the result.
The example above could be rewritten as such.