Print

[aaangeli]

Hi, I'm new to OTL, I've a little experience with TThread but now I'm interested on doing something better and for what I've seen and read OTL sounds very cool and professional. I'm not capable to upgrade UI during a thread pool because I know just a little of OTL.

My problem is simple and is this:
- I need to run one or more tasks in a thread pool
- I would like that each task during processing update a label in a form (with a counter) and also I need that the UI remains interactive

Now I'm thinking at something like this (note that DoSomestuff is only to do something different of classical Sleep!) :

Code: [Select]


procedure DoSomeStuff(task : IOmniTask; lb : TLabel; Max : Integer);
var
  ary : array[0..1024] of Byte;
  I : Integer;
  NF : HFILE;
begin
  NF:=FileOpen('file.bin',fmShareDenyNone);
  try
    for I:=1 to Max do begin
        lb.Caption:=IntToStr(I);
        FileSeek(NF,0,0);
        FileRead(NF,ary,1024);
    end;
  finally
    FileClose(NF);
  end;
end;

....

var
  waitGroup: IOmniTaskGroup;
begin
  GlobalOmniThreadPool.MaxExecuting:=2*System.CPUCount;
  GlobalOmniThreadPool.MaxQueued:=3;

  waitGroup := CreateTaskGroup;
  CreateTask(procedure(const task: IOmniTask) begin DoSomeStuff(task,Label1,3000000); end)
             .Unobserved
             .Join(waitGroup)
             .Schedule;
  CreateTask(procedure(const task: IOmniTask) begin DoSomeStuff(task,Label2,4000000); end)
             .Unobserved
             .Join(waitGroup)
             .Schedule;
  CreateTask(procedure(const task: IOmniTask) begin DoSomeStuff(task,Label3,5000000); end)
             .Unobserved
             .Join(waitGroup)
             .Schedule;

  waitGroup.WaitForAll;
end;

Obviously the labels are not updated and UI doesn't respond until all threads are finished because there is no application.processmessages at all or something similiar. Which is the best way with OTL to reach my goal? Have you got an example for something like this?

Thanks,
Davide

[Primoz Gabrijelcic]

See: http://www.thedelphigeek.com/2011/03/synchronize-comes-to-omnithreadlibrary.html

[aaangeli]

Thanks for the quick reply!

I'm doing some tests, Invoke is great, I've created a task and with .Run command I executed it, during the execution, I call in a timer event the Invoke method to update the UI and the UI is correctly updated.

Now I would like to do it with a thread pool. Is this possible? I'm trying it but the program always goes in deadlock, but maybe I'm doing something wrong. I need to execute more threads at the same time and then at the end of all of them I need to do some other stuff but during the threads elaborations I want to keep the UI updated.

Davide

Here is the code:

Code: [Select]

  TMyTask = class(TOmniWorker)
  protected
    FLb : TLabel;
    FMax : Integer;
    FTimer : TTimer;

    function Initialize: boolean; override;
    procedure DoTimer(Sender : TObject);

  public
    constructor Create(lb : TLabel; Max : Integer);
    destructor Destroy; override;

    procedure DoIT;
  end;

function TMyTask.Initialize: boolean;
begin
  Result:=inherited Initialize;

  DoIT;
end;

procedure TMyTask.DoIT;
var
  ary : array[0..1024] of Byte;
  I : Integer;
  NF : HFILE;

begin
  FTimer.Enabled:=TRUE;

  NF:=FileOpen('file.bin',fmShareDenyNone);
  try
    for I:=1 to FMax do begin
        Flb.Tag:=I;
        FileSeek(NF,0,0);
        FileRead(NF,ary,1024);
    end;
  finally
    FileClose(NF);
  end;

  FTimer.Enabled:=FALSE;
end;

constructor TMyTask.Create(lb : TLabel; Max : Integer);
begin
  FLb:=Lb;
  FMax:=Max;

  FTimer:=TTimer.Create(Nil);
  FTimer.Enabled:=FALSE;
  FTimer.OnTimer:=DoTimer;
  FTimer.Interval:=100;
end;

destructor TMyTask.Destroy;
begin
  FreeAndNil(FTimer);

  inherited Destroy;
end;

procedure TMyTask.DoTimer(Sender : TObject);
begin
  task.Invoke(procedure
              begin
                FLb.Caption:=IntToStr(FLb.Tag);
                FLb.Repaint;
              end);
end;


procedure TForm1.Button1Click(Sender: TObject);
var
  waitGroup: IOmniTaskGroup;
begin
  GlobalOmniThreadPool.MaxExecuting:=2*System.CPUCount;
  GlobalOmniThreadPool.MaxQueued:=3;

  waitGroup := CreateTaskGroup;

  // for now just only one thread
  CreateTask(TMyTask.Create(Label1,3000000))
             .OnMessage(Self)
             .Unobserved
             .Join(waitGroup)
             .Schedule;

  waitGroup.WaitForAll;
end;


[Primoz Gabrijelcic]

1) Don't use VCL components (TLabel, TTimer) inside a thread. VCL is not threadsafe and this will cause many problems.
2) Instead of TTimer, use OmniThreadLibrary timers (Task.SetTimer, Task.ClearTimer).
3) Don't call DoIt from the Initialize. Better way to achieve the same is:

Code: [Select]

  CreateTask(TMyTask.Create(Label1,3000000))
             .OnMessage(Self)
             .Unobserved
             .Join(waitGroup)
             .Invoke(@TMyTask.DoIt)
             .Schedule;


[aaangeli]

1) Ok I've removed direct VCL access in the thread for TTimer an TLabel. I look at some samples and now I'm using task.comm to send notifies at the form to update the labels.

2) I've tried to use Task.SetTimer but it is never fired. Look at code below: in the loop inside the DoIT procedure I can update the labels only if send manually the MSG_NOTIFIY every "n" (=10000) iterations. Paradoxically, in this case, VCL TTimer works. Maybe I'm doing something wrong.

3) As you can see in the code below, I can get the program work only clicking on button1 in which I simulate an execution without the pool. If I click button2, where I tried to schedule the tasks in a pool, I always get the program locked. Maybe there is something that I didn't realize well in waitforall and maybe there is a better way to achieve my goal.


The complete code is:

Code: [Select]

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, ExtCtrls, OtlComm, OtlTask, OtlTaskControl, OtlThreadPool;

const
  MSG_NOTIFY = WM_USER;

type
  TForm1 = class(TForm)
    Label1: TLabel;
    Label2: TLabel;
    Label3: TLabel;
    Button1: TButton;
    Button2: TButton;
    procedure Button1Click(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure Button2Click(Sender: TObject);
  private
    waitGroup : IOmniTaskGroup;
    wk1,wk2,wk3 : IOmniTaskControl;
  public
    procedure MsgNotify(var msg: TOmniMessage); message MSG_NOTIFY;
    procedure DoCreateTasks;
    procedure DoSometingAfterFinished;
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

type
  TMyTask = class(TOmniWorker)
  protected
    FLbNum : Integer;
    FMax : Integer;
    FCounter : Integer;

    function Initialize: boolean; override;

  public
    constructor Create(lbNum,Max : Integer);
    procedure DoTimer;
    procedure DoIT;
  end;

function TMyTask.Initialize: boolean;
begin
  Result:=inherited Initialize;

  if Result then
     task.SetTimer(1,500,@TMyTask.DoTimer);
end;

procedure TMyTask.DoIT;
var
  ary : array[0..1024] of Byte;
  I : Integer;
  NF : HFILE;

begin
  FCounter:=0;

  NF:=FileOpen('file.bin',fmShareDenyNone);
  try
    for I:=1 to FMax do begin
        FCounter:=I;

// !!! only un-commenting this instructions I got the label updated = the timer is never fired !!!
//        if (I mod 10000)=0 then
//            Task.Comm.Send(MSG_NOTIFY,[FLbNum,FCounter]);

        if Task.Terminated then
           Break;

        FileSeek(NF,0,0);
        FileRead(NF,ary,1024);
    end;
  finally
    FileClose(NF);
  end;
end;

constructor TMyTask.Create(lbNum,Max : Integer);
begin
  FLbNum:=LbNum;
  FMax:=Max;
end;

procedure TMyTask.DoTimer;
begin
  Task.Comm.Send(MSG_NOTIFY,[FLbNum,FCounter]);
end;

{TForm1}
procedure TForm1.Button1Click(Sender: TObject);
begin
  // execution without pool
  DoCreateTasks;
  wk1.Join(waitGroup).Invoke(@TMyTask.DoIt);
  wk2.Join(waitGroup).Invoke(@TMyTask.DoIt);
  wk3.Join(waitGroup).Invoke(@TMyTask.DoIt);
  waitGroup.RunAll;
end;

procedure TForm1.Button2Click(Sender: TObject);
begin
  // execution with pool
  DoCreateTasks;
  wk1.Unobserved.Join(waitGroup).Invoke(@TMyTask.DoIt).Schedule;
  wk2.Unobserved.Join(waitGroup).Invoke(@TMyTask.DoIt).Schedule;
  wk3.Unobserved.Join(waitGroup).Invoke(@TMyTask.DoIt).Schedule;
  waitGroup.WaitForAll;

  // I need all tasks finieshed here!
  DoSometingAfterFinished;
end;

procedure TForm1.DoCreateTasks;
begin
  wk1:=CreateTask(TMyTask.Create(1,3000000)).OnMessage(Self);
  wk2:=CreateTask(TMyTask.Create(2,4000000)).OnMessage(Self);
  wk3:=CreateTask(TMyTask.Create(3,5000000)).OnMessage(Self);
end;

procedure TForm1.DoSometingAfterFinished;
begin
  ShowMessage('Finished!');
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
  GlobalOmniThreadPool.MaxExecuting:=2*System.CPUCount;
  GlobalOmniThreadPool.MaxQueued:=3;
  waitGroup := CreateTaskGroup;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  waitGroup.TerminateAll;
end;

procedure TForm1.MsgNotify(var msg: TOmniMessage);
var
  S : String;
begin
  S:=msg.MsgData.AsArrayItem[1].AsString;
  case msg.MsgData.AsArrayItem[0].AsInteger of
    1 : Label1.Caption:=S;
    2 : Label2.Caption:=S;
    3 : Label3.Caption:=S;
  end;
end;

end.

Thanks,
Davide

[Primoz Gabrijelcic]

About timers - keep in mind that one task is always executing in one thread. IOW, a task is running single-threaded. If the task is executing very long function (DoIt) then the timer cannot fire.

Your Button1 handler doesn't wait for the tasks to complete - there is no waitGroup.WaitForAll. That's why it is behaving different than the Button2.

WaitForAll never completes because your tasks never stop working. TOmniWorker-based task must call Task.Terminate to terminate itself. (Or you can stop it from the outside by calling wk1.Terminate.)

Messages from the timer are never processed because your main thread (which is again only one thread and can do only one thing at once) is waiting in the WaitForAll.

Unobserved should not be used here as the result of CreateTask is stored in a variable (wk1 to wk3).

A better way to create your tasks would be:

Code: [Select]

procedure TForm87.Button1Click(Sender: TObject);
begin
  // execution without pool
  DoCreateTasks;
  waitGroup.RunAll;
end;

procedure TForm87.Button2Click(Sender: TObject);
begin
  // execution with pool
  DoCreateTasks;
  wk1.Schedule;
  wk2.Schedule;
  wk3.Schedule;
  waitGroup.WaitForAll;

  // I need all tasks finieshed here!
  DoSometingAfterFinished;
end;

procedure TForm87.DoCreateTasks;
begin
  wk1:=CreateTask(TMyTask.Create(1,3000000)).Join(waitGroup).Invoke(@TMyTask.DoIt).OnMessage(Self);
  wk2:=CreateTask(TMyTask.Create(2,4000000)).Join(waitGroup).Invoke(@TMyTask.DoIt).OnMessage(Self);
  wk3:=CreateTask(TMyTask.Create(3,5000000)).Join(waitGroup).Invoke(@TMyTask.DoIt).OnMessage(Self);
end;

[Primoz Gabrijelcic]

Here is a redesigned program:

Code: [Select]

unit Unit87;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, OtlComm, OtlTask, OtlTaskControl, OtlThreadPool;

const
  MSG_NOTIFY = WM_USER;

type
  TForm87 = class(TForm)
    Button1: TButton;
    Button2: TButton;
    Label1: TLabel;
    Label2: TLabel;
    Label3: TLabel;
    procedure FormDestroy(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure Button1Click(Sender: TObject);
    procedure Button2Click(Sender: TObject);
  private
    tasks: IOmniTaskControlList;
    procedure DoCreateTasks(runTask: boolean);
    procedure DoSometingAfterFinished;
    procedure MsgNotify(var msg: TOmniMessage); message MSG_NOTIFY;
    procedure TaskTerminated(const task: IOmniTaskControl);
  public
  end;

var
  Form87: TForm87;

implementation

{$R *.dfm}

type
  TMyTask = class(TOmniWorker)
  protected
    FLbNum : Integer;
    FMax : Integer;
    FCounter : Integer;
  public
    constructor Create(lbNum,Max : Integer);
    procedure DoTimer;
    procedure DoIT;
  end;

procedure TMyTask.DoIT;
var
  ary : array[0..1024] of Byte;
  I : Integer;
  NF : HFILE;

begin
  FCounter:=0;

  NF:=FileOpen('file.bin',fmShareDenyNone);
  try
    for I:=1 to FMax do begin
        FCounter:=I;

        if (I mod 10000)=0 then
            Task.Comm.Send(MSG_NOTIFY,[FLbNum,FCounter]);

        if Task.Terminated then
           Break;

        FileSeek(NF,0,0);
        FileRead(NF,ary,1024);
    end;
  finally
    FileClose(NF);
  end;
  Task.Terminate;
end;

constructor TMyTask.Create(lbNum,Max : Integer);
begin
  FLbNum:=LbNum;
  FMax:=Max;
end;

procedure TMyTask.DoTimer;
begin
  Task.Comm.Send(MSG_NOTIFY,[FLbNum,FCounter]);
end;

{TForm87}
procedure TForm87.Button1Click(Sender: TObject);
begin
  // execution without pool
  DoCreateTasks(true);
end;

procedure TForm87.Button2Click(Sender: TObject);
begin
  // execution with pool
  DoCreateTasks(false);
end;

procedure TForm87.DoCreateTasks(runTask: boolean);

  procedure CreateOneTask(id, repeats: integer);
  var
    task: IOmniTaskControl;
  begin
    task := CreateTask(TMyTask.Create(id, repeats))
      .Invoke(@TMyTask.DoIt)
      .OnMessage(Self)
      .OnTerminated(TaskTerminated);
    tasks.Add(task); // must be inserted before it is started!
    if runTask then
      task.Run
    else
      task.Schedule;
  end;

begin
  CreateOneTask(1, 3000000);
  CreateOneTask(2, 4000000);
  CreateOneTask(3, 5000000);
end;

procedure TForm87.DoSometingAfterFinished;
begin
  ShowMessage('Finished!');
end;

procedure TForm87.FormCreate(Sender: TObject);
begin
  GlobalOmniThreadPool.MaxExecuting:=2*System.CPUCount;
  GlobalOmniThreadPool.MaxQueued:=3;
  tasks := TOmniTaskControlList.Create;
end;

procedure TForm87.FormDestroy(Sender: TObject);
var
  task: IInterface;
begin
  for task in tasks do
    (task as IOmniTaskControl).Terminate(INFINITE);
end;

procedure TForm87.MsgNotify(var msg: TOmniMessage);
var
  S : String;
begin
  S:=msg.MsgData.AsArrayItem[1].AsString;
  case msg.MsgData.AsArrayItem[0].AsInteger of
    1 : Label1.Caption:=S;
    2 : Label2.Caption:=S;
    3 : Label3.Caption:=S;
  end;
end;

procedure TForm87.TaskTerminated(const task: IOmniTaskControl);
begin
  tasks.Remove(task);
  if tasks.Count = 0 then
    DoSometingAfterFinished;
end;

end.


[aaangeli]

Fantastic! It is exactily what I want and what I tried to do.

Now I've understood quite well a couple of things about threads, VCL and OTL.

Today I read that you are preparing the OTL documentation, I'll wait for it because I'd like to use OTL heavily in my projects and I'd like to know things better to not disturb you with this simple problems.

Thank you very much! I'll find a way to repay.

Davide

[Primoz Gabrijelcic]

Don't wait too hard - the documentation can take some time to prepare. Read my blog and ask questions here and all will be well.