04 March 2018

GfaWin23.Ocx Update 2.34

Another three bugs are fixed in this version. The SetPrinterByName needed maintenance because of the always growing need of memory with newer versions of Windows. The command failed with some printers that returned a lot of information in the GetPrinter() API. GFA-BASIC did not reserve enough memory and caused a buffer overrun. The EOF() function now also works with inline files, the ones that are stored in the :Files section of the program. The ListView.GetFirstVisible property now returns a ListItem.

About Version-numbering
The update gets FileVersion 2.34.1803 and still belongs to GFA-BASIC product version 2.3. The Build number now shows the year and month of the release. This is the first update with the new version structure. The DllVersion$ is 2.34 Build 1803 and indicates a release date of March 2018. A DLLVERSION structure always uses a 3 part format, major, minor and build. The VERSIONINFO resource on the other hand uses 4 part format. The GfaWin23.Ocx VERSIONINFO structure == 2.34.1803.0 and leaves the last part unused. This is the version showed in the File Properties dialog:

The ocx extension indicates a DLL with OCX-controls that are described using a type library. In contrast with the purpose of the extension, the OCX controls are not publically registered and are only available within in GFA-BASIC 32.

All OLE classes defined in the GfaWin23.ocx are private to the GFA-BASIC 32 application, each OLE class is implemented with its own command. This leaves no room for dynamically loading of other COM classes and automatic use of their interfaces (like VB). Third party COM classes can only be used when they implement a dual interface so they are accessible through a dispatch identifier. In GFA-BASIC these dual interface classes are supported through the use of CreateObject and the Object.property syntax. Unfortunately, calling an interface member (property/method) requires a two step process. First the caller must ask the server for an ID number and then use that number to actually invoke the member (property/method). Each dot operator requires this two step process and accessing dual interface members can cost quite some time when a command consists of many dots. For instance something like this: Object.List.Items(n).Text requires 6 calls to the COM-class provider. GFA-BASIC 32 provides a hidden optimization for automation objects created with CreateObject(). It caches all IDs in a hash-table the first time they are used. The next time a property/method is used it is looked up in the hash-table which is considerably faster.

24 February 2018

Non-bugs (1)– Events in a Form with controls

In the past years quite some (alleged) bugs have been reported. These reports range from editor-problems to compiler-errors to runtime bugs. Many of them have been addressed in a blog post or by a fix in an update in the GfaWin32.exe or GfaWin23.ocx binary. Still, there are issues that need some kind of an answer. In this series I’ll look at some of them.

Alleged Bug: A (single) button on a Form eats the form’s keystrokes.

This problem occurs when old-style GFA-BASIC programming is mixed with OLE Container/Control style of programming. Older GFA-BASIC (16-bit) programs often respond to messages in a message loop using GetEvent/PeekEvent. All messages are retrieved in a single place and dispatched based on a GB specific ID-value in MENU(1) or a Windows message value in MENU(11) or its alias _Mess. This is no problem as long as there are no controls on the window. 

For instance, the well-known OpenW command behaves the same as in older GFA-BASICs, that is until a control is created in its client area. When the window contains a control (OCX) it starts behaving as an OLE Control Container according COM-specifications. In GB32 the concept of simple controls no longer exists. Controls are now OLE controls wrapped in COM objects and behave according strict OLE-rules. One of the consequences is the change in keyboard handling. Now, the focus can no longer be set to the client area of the window, because the client area is now a control container site and has COM defined responsibilities; “An OLE Control Container implements keyboard handling by calling specific methods on a predefined COM interface that the controls supports.” In practice this means that all keyboard messages are send to controls and only controls can have the focus. This makes sense because an OLE control containers must support default and cancel button handling, mnemonic handling, and tab handling, including maintaining tab order.

As an example a snippet that won’t work because the old-style of message handling no longer works. The following is ‘impossible’, because the OpenW 1 is a Form (control container) with a single button OCX. All keyboard messages are (as should be) dispatched to the currently active control, which is - in all circumstances - the single button.

OpenW Center # 1
Ocx Command cmd1 = "Click", 200, 100, 190, 26
  // ~SetFocus(Win(1))  ' This doesn't help
  Select _Mess
    Print "WM_KEYDOWN"
  Case WM_CHAR
    Print "WM_CHAR"
Until Me Is Nothing

Because the button is the only OCX on the Form it will regain focus over and over, how hard you try to shift it back to the window.
If you like a Form to respond to keyboard events don’t use controls or use a separate child Form Ocx next to the other controls.

A Bug: A button on a ChildW form eats the form’s mouse clicks.

In the same situation as above, a window with a button (or a toolbar) the Form’s mouse clicks might be passed on to the button or toolbar. The bug presents itself in ChildW forms in a MDI-application, not in single window.

08 February 2018

A Manifest does not guarantee Visual Styles

This is part 2 of the problems that arise when using manifest files. In the first part: Update doesn't load manifest file we saw how to force Windows to load the common controls version 6, rather than to default to version 5. A manifest could be included as a resource, but it can also be an external file in the same directory as the executable. External manifest files do overrule the embedded manifest resource. To be sure the Windows loader recognizes the manifest file – for GfaWin32.exe - the timestamp of the exe must be newer than the manifest’s last modified date.

But there is more. A correct manifest file and correct dates do not guarantee that version 6 is actually loaded. For instance the following must be taken into account

  • If there are spaces in the .exe name (e.g. the exe is called “this is executable.exe”), the manifest file (“this is executable.exe.manifest”) would not work – Common Controls were not displayed correctly; however, if the spaces are replaced with underscores in both files (“this_is_executable.exe” and “this_is_executable.exe.manifest”) they did.
  • An exe seems to respond better when a manifest is embedded as a resource than as a stand-alone file. With an embedded resource you have some sort of guarantee that the manifest is applied and that the common control dll is loaded.

How are controls painted?
The comctrl32.dll – both version 5 and 6 - is now responsible for the drawing of all controls, even the standard ones that used to come in user32.dll. This makes it easier to paint the controls in a  unified style, where the controls from version 6 are painted according the current theme selected by the user. Common control version 5 is not affected by the theming, all controls are painted in a default style coming with a particular Windows OS. So, even without version 6 controls slightly differ from one Windows version to the other.

When the controls from version comctrl32 version 6 are used controls can look different on every other user’s screen. The developer hasn’t much to say in this. Borders, 3D effect, text-color, and background color can all be changed by the user by selecting a new theme. After selecting a new theme the new style takes effect immediately and the look of the controls change accordingly. What’s left for the developer is defining which parts of the controls are to be used. For instance, a Command button can have a basFlat or basThreeD style, but how the resulting control looks like can not determined before hand. Some themes draw the flat and 3D style exactly the same.

Older Windows allow disabling Visual Styles
In older Windows versions the user (or system manager) can overrule the use of common controls version 6. Especially with Windows 7, Vista and XP the user can disable the Visual Styles completely in Control Panel, or only for the particular application in the Properties – Compatibility tab of the executable. So, even when your application depends on version 6, an older OS might allow to disable certain features.
Starting with Windows 8 this is no longer the case; the theming is applied to every GUI element and the user cannot disable the theming.

So, for older Windows versions it is possible that a compiled and manifested exe executes without applying theming. Although your program does load comctrl32.dll v6 it might look old-school style (but hey, it is what the user wants). You don’t know what is actually painted in this situation. In fact, with all these different OS versions and personalization settings you can not predict at all how your program will look like exactly.

Recommendations for using controls

  • Use the newer common control version 6, this ensures painting according the default theme of each new Windows update and makes your app look more up-to-date.
  • Use default settings for all controls, don’t change text color or background color, these parts are controlled by the theme.
  • Add a resource manifest to the compiled exe, since using a manifest file might give you trouble.

Note Despite these recommendation GfaWin32.exe comes with a manifest file so that users can delete the manifest and use old-style controls for their legacy programs. Do remember that users can add a manifest file to your compiled exe themselves and thereby change the overall look of your app. 

Form Editor - Differences in painting OCX
GfaWin32.exe comes with a manifest file (stripped to only load common controls version 6) and the controls are displayed in the themed style see the previous post. The IDE itself uses pure API functions to create controls, but the Form Editor behaves quite different. The Form Editor uses specific COM design-time interaction between the host (Form) and the OCX controls. The runtime GfaWin23.ocx provides two different code-paths for communication between the host (Form) and OCX controls: a run-time and a design-time handling. There might be a discrepancy between these modes. The only way you can tell if an OCX behaves as expected is by checking the result by running (F5) the program. Then test it as a compiled exe. (You can use Launch Exe from the Project menu.) There should be no reason why a program RUN in the IDE behaves different from a stand-alone exe. The exact same runtime code is executed.

Note - There is a little catch. There is a flag in the runtime (gfawin23.ocx) that is set when the IDE is active and theoretically it is possible that the runtime behaves somewhat different. In the huge COM-related disassembly it is hard to determine what it actually does. Only a stand-alone exe that gets all the attention from the runtime without keeping an eye on the IDE. This might explain the discrepancies people noticed.

Is Themes Enabled
When problems occur it is possible to inquire about the theming state of the executable. First of all you need to check whether the program is actually using common controls version 6. This is accomplished by testing CommCtlVersion  property of the Screen Object:

If Screen.CommCtlVersion >= 6
  MsgBox "Common controls version 6 loaded."

If the required common controls version is loaded we also need to know if the application is actually themed (on older Windows). The controls need to be painted using the current theme. For this purpose there are some additional APIs located in uxtheme.dll (which comes with Windows XP and later). Two of them seem to provide information about the current theming state of the application, in particular IsThemeActive() and IsAppThemed(). However they come with limitations.

  • IsThemeActive() returns TRUE when Visual Styles are enabled for the user. It is a user setting that can be changed in Control Panel for Win 7 and lower. On Windows 8 and above this function always returns TRUE. IsThemeActive() isn’t very useful, it is included in the IsAppThemed() API.
  • IsAppThemed() checks to see if theming (Visual Styles) are on, which is the same as IsThemeActive(). In addition it checks the existence of a manifest file and also the Compatibility tab of the executable’s Properties. If all three conditions are true IsAppThemed() returns TRUE.

IsAppThemed() returns true if the theming is applied to the application. If also Screen.CommCtlVersion >= 6 (thus manifest is loaded) the application uses theming for common controls version 6.  With the following code snippet you can test the current theming state of the common controls version 6.

Declare Function IsAppThemed Lib "UxTheme.dll" () As Boolean
Trace Screen.CommCtlVersion >= 6 && IsAppThemed() ' True/False

This concludes the posts on manifests to enable themed common controls. If you have any comments or question please use the Comments section below.

For more information see https://www.codeproject.com/Articles/620045/Custom-Controls-in-Win-API-Visual-Styles

04 February 2018

Update doesn’t load manifest file

Problem: After installation of the latest update GFA-BASIC 32 shows old-style common controls. There is only one reason for this failure: the manifest file isn’t loaded.
After starting GFA-BASIC 32 the differences are clear from the tabs in the sidebar:

The left image shows the IDE without common controls 6.0 support as opposed to the right image that shows new style controls.

GB External Manifest
GFA-BASIC 32 uses an external, side-by-side, manifest. The manifest is used to enable the common controls version 6 for the GfaWin32.exe. The manifest could have been added as a resource, but not all GB32-programs support the windows common controls version 6. In general, an external manifest should be as good as a resource enabled EXE. However, modified external manifests aren't applied always properly or aren’t applied at all. Windows doesn’t always recognize a new manifest and it requires a reboot to get a new manifest loaded. But, not even that solves the problem for everybody. Some never suffer from this problem, others seem not to get rid of it. At first I couldn’t reproduce this behavior, let alone I could solve it ...

After some research I came across the following pages blog #1 and blog #2 that explain this behavior. Coupled with the bits and pieces I further read on the internet I hope to have found an answer and it is not entirely the same as described.
The Activation Context Cache
To improve CreateProcess performance, manifests are cached in the Activation Context Cache. Its not the manifest-file that gets cached, but a note that some EXE requires a Manifest-file. The cache stores the full path of the exe and the last modified time of the exe file. To force a reload of the manifest-file the last-modified date of the EXE must be newer than that of the manifest file.But that doesn’t always solve the problem, the cache’s entry must be removed first.

Re-enable the manifest
The binaries from the GFA-BASIC 32 Update are newer than the manifest file and oddly the controls are still displayed in the old-style. Seems the theory doesn't apply entirely, otherwise the manifest would be loaded properly. Probably the cache must be cleared first (reboot) because it contains data that prevents the loader from applying the manifest. We need to invalidate the cache’s entry for GfaWin32.exe. One way to proceed is like this.

  1. Make sure the modification date of the EXE is more recent than the last-modified date of the manifest file (more about this in a minute).
  2. Start GfaWin32.exe without a manifest present to invalidate (or remove) the entry in the cache that applies to the GfaWin32.exe. For example. rename the manifest to something like GfaWin32(1).exe.manifest. Old-style controls will be displayed.
  3. Reboot the computer.
  4. Rename the manifest to its previous name: GfaWin32.exe.manifest
  5. Start GFA-BASIC 32 (and keep your fingers crossed).

Only way I got it right
This process described above might fail again, at least with me it didn’t work all the time. The only way I could make this work is by re-setting the last-modified date of the GfaWin32.exe. I had to explicitly apply the Touch # command on the GfaWin32.exe to have the manifest-file loaded and applied.

Of course, as a programmer you might like to write your own Touch utility, but you cannot Run it inside the IDE to access the GfaWin32.exe you are running from. For convenience reasons you could download this Touch tool and copy it to the GFA-BASIC’s Bin directory. Before running the tool make sure GfaWin32.exe isn’t running.

New Update Feb 2018
In Download section you can also find the new time stamped files included in Feb 2018 update zip files. The manifest file itself has changed also, it is now simple file as discussed below. Hope it fixes the problem.

Add Manifest-resource to your own EXE
When researching this problem I tried different manifest files. I found the manifest file generated from mt.exe as described in Using mt to include manifest on this blog very useful. The manifest only specifies the dependency on the common controls 6.0. The manifest is very clean and isn’t dependent on version-info and the name of the exe. You get this same manifest-file as a resource in your own EXE when you check the Add manifest resource checkbox in the compile-to-exe dialog box. You can also add it as a ‘side-by-side’ file and put it in the same directory as your exe.

28 January 2018

Update January 2018

It took a while, but this months update fixes some fatal crashes and provides a few new features. Go to the Download page for more details. Be sure to read the statement on Donations first.

Due to fatal errors the update had to be available as soon as possible. It was prepared in a great hurry and many almost-ready things didn’t make it into this version. So, the update might come with a few annoyances, some little, some cosmetic. Expect more updates to be available in the coming months.

Most changes and additions are mentioned in the ReadMe24.rtf. I hope to discuss some of the new features in coming posts.

The only new feature I like to mention here is Proc PeekView, which allows you peek into a procedure without actually moving to that procedure. Hoover the mouse over a procedure name, either in the editor or in the Procs tab in the sidebar, and a scrollable tooltip will appear. The tooltip shows the procedure header and gives a hint on the definition of the procedure. When you scroll the mousewheel more lines will appear and gives you a peek view of the procedures code lines. No more jumping around. Here is an example.


29 November 2017

The initialization of the Printer object

Each time you press F5 to run a program the Printer object is reinitialized. As part of the initialization process the program obtains information about the current default printer to prepare the Printer object for use in your program. The same happens in the start-up process of a compiled stand-alone program. When there is no default printer, or a corrupt printer-driver, the Printer’s properties and methods are undefined. You may use one of the Printer‘s properties .DeviceName, .DriverName, or .Port to obtain information about the actual printer associated with the Printer object. For instance, to check for a valid initialization:

Dim fValidPrinter? = Len(Printer.DeviceName)

When the Printer object needs to be associated with the current default printer at all times, you can compare the device-name against the current default printer returned from App.PrinterName(0). Before the printing process actually starts check for a change and re-initialize the Printer object to the new default printer:

' Before printing
If Printer.DeviceName != App.PrinterName(0) ' change of default?
  SetPrinterByName App.PrinterName(0) ' Re-initialize to default 
' Start printing

Note - Keeping track of the current default printer this way will not preserve any changes you made to the Printer object.

Available printers and their info
The App object provides properties that enables you to gather information about all the available printers on the system. For instance, to find a printer that has the Orientation set to portrait, starting with the first non-default printer:

' Iterate over available printers
Local i%, PrtName$
For i = 1 To App.PrinterCount    ' index = 1
  PrtName =  App.PrinterName(i)
  Debug PrtName : Debug " Info: "; App.PrinterInfo(PrtName, "")
  If App.PrinterInfo(PrtName, "Orientation") == 1
    Debug " > Printer in portrait mode"

The current default printer is returned by the App property PrinterName(0), with the index = 0. To retrieve all printer devices iterate over App.PrinterName(i) starting with index 1.

The PrinterInfo property retrieves detailed information of the available printers. PrinterInfo is a small wrapper for the GetPrinter() API using the PRINTER_INFO_2 structure to retrieve the information (see SDK). All member names of PRINTER_INFO_2 structure can be passed as an argument to PrinterInfo(PrinterName, membername$), which returns a Variant with the member’s value. Depending on the member-name, must be one of the below, the Variant contains a string or a Long.


In addition, PrinterInfo returns a few DEVMODE settings;


Finally, in case you pass an empty string - PrinterInfo(PrinterName,””) – it returns one string combining the settings of 'Server', 'Printer', 'Share', 'Port', 'Driver', 'Comment', 'Location', 'Port', 'SepFile', 'PrintProcessor', 'Datatype', and 'Parameters' as key/value pairs.

Printer Object initializing
By default the Printer object is initialized for the default printer. Using the Printer object lets you retrieve or set the settings of the default printer. Any printer output is based on the Printer object’s settings. To change the printer associated with the Printer object GB provides four commands:

Set Printer = cd                ' assign a CommDlg Object
SetPrinterByName "PrinterName"  ' specify a device name
SetPrinterHDC hdc               ' based on a printer device context
SetPrinterHDNHDM hDevName%, hDevMode% ' pass memory handles

The SetPrinterHDNHDM is at the heart of all printer changing commands. The other commands act basically all the same: they retrieve the the DEVMODE and DEVNAME structures from the input parameter and then call SetPrinterHDNHDM. The DEVMODE and DEVNAME structures provide all the information needed to properly initialize the Printer object. The CommDlg object, for instance, holds a DEVMODE and DEVNAME structure after having invoked the ShowPrint dialog box.

The Printer’s default initialization at start up requires these structures as well, and they are obtained by calling the PrintDlg() common dialog API. The PrintDlg() API function only requires two flags:

  • The PD_RETURNDEFAULT flag is used to retrieve information about the default printer without displaying the Print dialog box.
  • The PD_NOWARNING flag prevents the warning message from being displayed when there is no default printer.

At start-up the PrintDlg() is asked to return information about the current default printer without displaying the dialog box and without displaying a warning if the printer is missing. The existence of the PD_NOWARNING flag suggests it is possible (maybe there are no printers at all).
PrintDlg() may also fail because of a corrupted printer driver, or some other reason causing a delay in execution. It isn’t guaranteed that the PrintDlg() will always succeed. Consequently, it isn’t guaranteed that the Printer object is always properly initialized. When PrintDlg() fails GB initializes the Printer object’s private data to zero. This way the global Printer object is at least ‘not Nothing’, but its behavior is undefined because of the lack of actual device information.

Note - The SetPrinterHDNHDM command is useful when combining API functions and the Printer object. It takes 2 handles (hence HDN and HDM) to unlocked global-memory containing the DEVNAME and DEVMODE structures respectively. The hDevName and hDevMode arguments are handles to moveable global-memory objects allocated using GlobalAlloc(GMEM_MOVEABLE, sizeof(DEVMODE/DEVNAME)). They must be unlocked before passing to SetPrinterHDNHDM.

28 September 2017

CreateObject caching (.NET example)

Thanks to a little experiment using .NET classes from GB, I discovered a bug in the caching of dispids for IDispatch objects created using the CreateObject() function. (An update of the OCX runtime is available.) For demonstration purposes I’ll show you code accessing .NET classes I used to test caching. Caching is only available for objects created by CreateObject. I discussed the peculiarities of CreateObject in an earlier blog and if you need to freshen your memory you might want to check it out.

CreateObject and caching
The CreateObject function not only differs from VB/VBScript in creating a COM class and connecting to the IID_IDISPATCH interface, but it also sets up a Hash table to store the dispatch identifiers of the properties and method names. Each(!) object created from CreateObject is encapsulated in another hidden IDispatch object. The Object returned by CreateObject is a pointer to this hidden GB-provided COM object. The hidden object implements all 6 IDispatch functions, but only provides custom functions for GetIDsOfNames and Release. The other four (QueryInterface, Add, GetTypeInfoCount, and GetTypeInfo) are simply routed to the automation object directly. The hidden object provides the caching.
Only objects created with CreateObject are affected, that is if the class is not a GFABASIC32 Ocx, StdFont, or StdPicture type. When these terms are met, the CreateObject function creates an empty hash table and stores it in the encapsulating IDispatch object. When a property or method is executed for the first time, the identifier of the name is looked up using IDispatch.GetIDsOfNames function and saved in the hash table for future use.

Why caching is useful
The compiler converts the execution of a  property or method into a call to a runtime-library function which performs the actual execution of the property/method. The property/method is directed to the IDispatch.Invoke function. However Invoke does not accept named properties/methods, but only numeric values identifying the property or method. This integer value is called dispid. (You can retrieve a dispid yourself using the _DispId() function.) Before Invoke is executed the dispid value associated with the property/method name must the obtained by calling IDispatch.GetIDsOfNames function. Calling GetIDsOfNames each time before calling Invoke, decreases performance. This is especially true for out-of-process servers; non-DLL servers, DLLs are in-process servers, loaded into the process memory. Communicating with an out-of-process server is time consuming. Even more when the communication requires two steps, obtaining the dispid and executing the function. Once a property or method is used and its disp-ID value is known, it is logical to store the ID for future use in an easy accessible Hash table.

The encapsulating object
All IDispatch properties and methods are executed by the same runtime-library function, which is unaware of cached IDs. The GetIDsOfNames function is executed always before calling Invoke. However, the hidden Object that encapsulates the automation object, supports a custom GetIDsOfNames and first searches the hash table for the name. If it finds the name it has the ID value available directly. If the hash element isn’t found, it calls GetIDsOfNames on the external COM object and then stores the ID in the hash. When the Object goes out of scope, the custom Release function erases the hash table and the encapsulating object is freed from memory.

Example using .NET
The following code creates a .NET classes that support an IDispatch interface. The classes used are located in the mscorlib.dll, which is loaded into the process-space of the GB-application. The performance gain measured is about 15%. It will be better for out-of-process servers, the example is simply as a test for caching.

' ArrayList coclass supports multiple dual intefaces:
'  IList, ICloneable, _Object, ICollection, IEnumerable
' Togeher their exposed properties/methods make-up
' the ArrayList's class interface.
Dim oArrList As Object, v As Variant, i As Int
Set oArrList = CreateObject("System.Collections.Arraylist")
' IList
oArrList.add 3                ' IList.Add
oArrList.add 4
oArrList.add 4
Trace oArrList(0)             ' IList.Item(index)
Trace oArrList.Contains(3)    ' ILIst.Contains
' ICloneable
v = oArrList.Clone            ' ICloneable.Clone
' v is copy of ArrayList:
Trace v
' _Object interface
Trace v.ToString              ' _Object.ToString
Trace v.GetHashCode           ' _Object.GetHashCode
Set v = Nothing
' ICollection
Trace oArrList.Count          ' ICollection.Count
Trace oArrList.SyncRoot       ' ICollection.SyncRoot
' IEnumerable
For Each v In oArrList
  Trace v

' Queue collection
Dim myQ As Object
Set myQ = CreateObject("System.Collections.Queue")
myQ.Enqueue "Hello"
myQ.Enqueue "World"
myQ.Enqueue "!"

26 August 2017

A little array magic

Without going into a formal description of an array, we simply state an array stores multiple values of the same type in contiguous memory. In code an array is recognized by a variable name followed by parenthesis, either with or without indices. Like any other variable an array should be declared before it can be used. (Declaring a variable introduces a variable to the compiler.) Generally, a declaration specifies a name and a type. In case of an array the declaration may include values for lower and upper boundaries up to 7 dimensions.
Array declaration and dimensioning
An array declaration always results in the creation of an array-descriptor. For a global array the descriptor is added to the program’s global data section and for a local array the compiler inserts code to allocate an array descriptor dynamically.
' Declaration and allocation separated:
Global Dim a() As Long  ' adds descriptor to data
ReDim a(6)              ' code to allocate memory

' Declaration and allocation at once:
Global Dim b(3, 1) As String
The second declaration forces the compiler to add a descriptor to the global data and to generate code to allocate memory. It is the exactly the same as Global Dim b() As String : ReDim b(3,1).
A local array variable declaration is handled differently from a global declaration. First of all, the array is not assigned a static descriptor by the compiler. The declaration of the local array let the compiler insert code to obtain (or allocate) an array descriptor dynamically when the procedure is executed. The pointer to the descriptor is stored in a hidden local memory location on the stack of the procedure. Then the address of the descriptor is passed to the same ReDim to allocate memory for the array elements.
Proc LocalArr()  ' Naked forbidden
  Dim dum$       ' prevent compiler bug
  Dim h()        ' allocates a descriptor
  ReDim h(4, 5)  ' allocates memory for descriptor
  Dim v(3)       ' 1-step: descriptor + memory
EndProc          ' destruction for h() and v() and dum$

Local arrays have the same anatomy, but they have no descriptors in the global data-section. Both the descriptor and the memory are allocated – from the heap - when the subroutine is executed. Room is reserved on the procedure stack for a (hidden) pointer to store the address of the descriptor. Later this pointer is necessary to clean the local stack and call the array-destruction code when the subroutine is left.

Local Array Destruction
Local array destruction is part of the termination handler of the procedure, that is if it has one. A Naked procedure doesn’t include termination handlers; the procedure needs to clear pointer variables manually (= the developer). However, a local array cannot be destructed manually, there is no statement to do so. The obvious Erase would only release the data memory, not the array-descriptor, leaving it on the stack. Eventually, the stack might overflow when the procedure is executed repeatedly. 

  • An array cannot be destroyed explicitly, Clr doesn’t work with arrays (and hashes).
  • Local arrays are not allowed in Naked procedures.Naked prevents the compiler from insertion of destruction code for all pointer variables (String, Variant, Object).

Be aware of two possible problems
When a subroutine contains only one or more local array variables, without any other local variables of pointer types (String, Variant, Object), the compiler ‘forgets’ to insert the array-destruction code at all. This is a bug. In this specific situation it is necessary to force the compiler to add array-destruction code. This requires the introduction of another dynamic data-type that requires destruction code. A local String is the easiest solution as is demonstrated in the example above.( A bug still waiting for resolving ….)

The other problem involves ReDim, which - unlike Dim - does not default to the Option Base setting. Instead, ReDim always uses 0 as the lower bound of the array. When Option Base 1 is the default setting for your application, you need to use ReDim ar(1 .. x) explicitly, rather than ReDim ar(x).

- Important note on a Hash
A local Hash isn’t destroyed automatically as well (Naked or otherwise). Clr cannot be used and there is no way to force the compiler to insert Hash destruction code. All local Hash variables must be released manually using Hash Erase. You might want to use Static Hash for local variables. A Hash is a (relative) time consuming type, all entries of a Hash are released one by one. Static preserves the contents and prevents time consuming destruction.

Global Array destruction
GB implements hidden destruction for releasing arrays.  A local array is destroyed on exit and a global array when the program is terminated. For a global array the descriptor is static and part of the global data-section and is an inherent part of the program. After a program exits (either as an EXE or in the IDE) the global data-section simply disappears and the descriptors with it. In case of an EXE-process all memory is released to the OS, and in the IDE the global data is destroyed after ending the program (RUN). There is no cause for memory leak on global arrays.

Anatomy of an array
In GB32 an array is described using a variable name, a descriptor, and a piece of contiguous memory to store the array data. When the compiler hits on a global array declaration it will create a mapping between the variable name and an array-descriptor stored in the global data section. This is true for in-memory compiling and when an EXE is created. A local declaration introduces a mapping between a hidden local pointer variable (32-bits pointer) and the name. The hidden variable stores the pointer to the dynamically allocated array descriptor.
An array-descriptor is a structure defining the attributes of an array. This ArrayDesc - structure is defined like this (note how the last member reserves LBound/UBound information for a maximum of 7 dimensions):

Type ArrayDesc
  -Int    Magic         ' "arry" or "ArrY"
  -Int    ptype         ' vtType (internal const)
  -Int    size          ' size of datatype
  -Int    dimCnt        ' number of dimensions
  -Int    dimCnt2       ' # of dimensions     == IndexCount
  -Int    paddr         ' address of data     == ArrayAddr()
  -Int    corr          ' correction value
  -Int    paddrCorr     ' void* addrCorr;
  -Int    anzElem       ' number of elements  == Dim?()
  -Int    sizeArr       ' size in bytes       == ArraySize()
  -Int    Idx(7 * 3)    ' == LBound()/UBound()

For global and static arrays an instance of this structure is stored in the global data section. For local arrays the structure is allocated dynamically. Important to realize is that every declaration (Dim/Global/Local/Static) of an array immediately results in an array descriptor dimmed or un-dimmed. The values of the structure members determine the status of the array. The Magic member is for internal use, although it perfectly well indicates if an array is empty – Erase-d or an empty declaration. Other members can be retrieved using  the following functions.

FunctionMember ArrayDescDescription
Dim?(a()) anzElem (element count) Returns the number of elements in the array. Erase clears this value (sets to 0). One way to determine if an array has been ‘dimmed’.
IndexCount(a()) dimCnt2Returns the number of dimensions. Returns 0 when not ‘dimmed’. Another way to determine if an array is empty.
ArrayAddr(a()) paddrReturns the memory address of the first element of the array. Returns 0 if erased or not ‘dimmed’. Can be used to determine if an array is empty.
ArraySize(a()) sizeArrReturns the size of all elements in bytes. Returns 0 if array is empty.
LBound(a()[,i=1]) Idx[]Returns the lower bound for a dimension (default is 1). Raises an error when an array is empty.
UBound(a()[,i=1]) Idx[]Returns the upper bound for a dimension (default is 1). Raises an error when an array is empty.
  • Only LBound and UBound cannot be used to inquire for an ‘un-dimmed’ array.
  • For the special case OLE Automation array-type ParamArray, LBound and UBound return 0 and -1 respectively; these functions do not raise an error (VB compatibility). The ParamArray datatype is in fact nothing more than a Variant containing an OLE/COM SafeArray.
Functions and statements that do not apply to arrays
An array variable is treated differently from any other variable type. The array’s variable name cannot be used in any other GB32 functions and statements as other variables can. For instance, the Clr a() statement is forbidden, ArrPtr() function does not return the location of the array’s variable name, but the location of the array-descriptor instead. You cannot use Pointer to redirect an array variable name to another descriptor. TypeName(ar()) cannot be used to obtain the data type of the array. Etc.
  • Generally, all GB-functions and statements that use a variable name as an argument are forbidden for arrays.
When the compiler refers to an array it refers to the descriptor directly. The compiler doesn’t preserve a mapping between the array’s variable name and a particular location of a pointer as it does with Strings for instance. The generated code simply doesn’t ‘know’ the array name anymore, only the location of descriptor.
There is only one runtime function that accepts the address of a (local hidden) pointer containing the address of an descriptor: CLEARARR() the local array destructor. This function cannot be invoked manually – not even when using assembler, because the address of the hidden variable is unknown. Asm lea eax, ar will not work, it still returns the address of the descriptor.

02 August 2017

StatusBar fix (August 2nd Update)

Fixing one bug often leads to the introduction of another. The June update introduced a bug in the runtime in the Panels.Remove method. This is fixed and the most recent update of the runtime – gfawin23.ocx – is now version Although I like to step away from incrementing build-numbers, I decided to give it a new build because nothing new was added, only a bug is fixed.

StatusBar redraw problem
During testing I noticed the following. In some circumstances the StatusBar isn’t updated or redrawn to reflect the new situation with a panel removed. This is not GB32 bug, but it is related to the common controls. The StatusBar is only completely reset when it receives a WM_SIZE message. Usually, this message is forwarded by the Form, the parent of the control, when it processes the WM_SIZE message. When the Panel is removed the code does not send or post this message and the Form will never receive the message. The following sample code solves this problem, where sb is the name of the StatusBar Ocx.

' Remove a StatusBar Panel:
"Panel1" ' WM_SIZE recalculates and updates: PostMessage sb.Parent.hWnd, WM_SIZE, 0, 0

The lacking update and redraw is a known problem for more common controls. Usually this can be solved by posting a WM_SIZE message to the parent (without any values in the wParam and lParam). The preferred solution would be invalidating the control followed by a call to the UpdateWindow() API. However, this doesn’t recalculate and reset the panels. The status bar is simply redrawn using the ‘previous’ or current settings. This makes sense when you think about it. If the status bar would recalculate its contents each time it has to redraw, it would provide a serious performance penalty in redrawing a Form when it is manipulated. The common control needs to be told explicitly to adjust to the new situation.

28 June 2017

Update June 2017

I'm still fixing bugs and I'm still using GB32 on a daily basis. Sometimes fixes are ready to go public, this is such a day. The most important update is the GfaWin23.Ocx runtime, but the IDE also got a swing. Anyway go to the Download page for more information.
Don't hesitate to mail me when you have something to say about these updates. You can use the comments section below this post or mail me directly. You know where to find me.

Also: I updated the blog on UNICODE conversion. There was a bug in the Ansi To Unicode conversion function WStr().

25 May 2017

New Group/Mailinglist

A new mailinglist has been created: the  GFAGroup. This works the same as the previous Mailinglist that went down at the beginning of this year.
This mailinglist/group is provided by Google and thus requires a Gmail-account. After you have signed in with Google using your Gmail-account you can join the list. Although it is a group, it works much in the same way as the previous mailinglist. Questions and answers are posted through e-mail.

12 May 2017

ANSI, UNICODE, BSTR and converting

Update 28-06-2017 - Conversion from Ansi to Unicode: WStr() function.
For some reason the WStr() routine contained a stupid bug (that I now have fixed).

More info: The number of bytes to read from a BSTR-address was wrong. GFA-BASIC always uses the SysAllocStringLen(Null, lenbytes) when allocating COM String memory. The BSTR returned is preceded by a 32-bits value specifying the BSTR's number of bytes, not the number of characters! This is exactly the value needed when reading the BSTR-bytes into a String datatype using StrPeek(). So, the function should have been: StrPeek(BSTR, {BSTR-4}), see the updated function below.

Another point of confusion was about the number of terminating null-bytes that WStr() returned. The StrPeek() function in WStr() only copies the UNICODE characters from the BSTR to a String, without the two null-bytes that secretly follow a BSTR string. As a result, the UNICODE characters copied to the String datatype are followed by only one (1) null byte; the terminating null byte that each String secretly gets.
When a String of UNICODE characters is to be passed to a Wide API function, two null-bytes must be added 'manually'.
w$ = WStr("GFABASIC") + #0#0 ' assign two nullbytes

The post as it was:
In the previous post I discussed UNICODE versus ANSI in the ANSI-based GFA-BASIC. Basically, GB doesn’t support UNICODE because it expects 1-byte characters where strings are used. In UNICODE each character occupies 2 bytes and allows more than 256 characters. Conversion ANSI to UNICODE is ok, but conversion from UNICODE to ANSI might lead to a loss of characters with a value above 256. But there is more: Variants and BSTRs.
The introduction of COM in GB required the provision of a new data type, the Variant. The Variant is a 16-byte data type that holds data and a value that specifies the type of that data(LONG, CARD, DOUBLE, etc). A Variant can also be used to store (safe-) arrays, a specific COM array type, and BSTRs, special UNICODE strings. So to understand the String and BSTR/Variant in detail ….

How a String is stored
Because a BSTR is much like a GFA-BASIC String data type, I’ll first tell how a GB String is stored. You could skip this part if you already know.
Declaring (Dim) a String-variable introduces a name for a location. The String-variable itself requires four bytes to store a pointer to dynamically allocated memory for the characters. The declaration and assigning a location is handled by the compiler, the rest happens at runtime: assigning or initializing. When the String-variable is initialized a call to malloc() reserves memory for all its characters with an additional 5 bytes. The first 4-bytes are reserved to store the length of the string and the last byte for the null-byte (not included in the length value). After allocating and copying the characters, the address of the first character of the string is stored at the variable’s location, a 32-bits address or pointer.
Global a$       ' 32-bits location(=0) in data or stack
a$ = "GFABASIC" ' assign pointer (address) to location
l = Len(a$)     ' address <> 0 return length {address-4}
Clr a$ : a$= "" ' free memory, set locations to 0
- String in memory: [xxxx|cccccc…c|0]
- Initially, the variable is a null pointer, the contents of the variable’s location is 0.
- String variable points the address of the first character c.
- Length is stored in position address – 4, and does not include the terminating zero.

Obtaining the string’s length is a 2-step process. First the variable is tested for a non-null pointer and than the value of the preceding 4 bytes (string-address – 4) is returned.
- Clearing a string (or assigning an empty string “”) will free the allocated memory and reset the variable’s contents to 0.

GB does not provide a data type BSTR, but it provides limited support of hidden BSTRs to pass and obtain BSTR-strings to and from COM objects. GB handles the conversion and memory allocation for BSTRs, but it does not provide string-manipulation functions for BSTRs, or even BSTRs in Variants. More on this below.
BSTR variables are always temporary, hidden local variables used to communicate with COM properties/methods that take or return BSTR arguments. These hidden BSTR variables are always destroyed when leaving a subroutine. Even the Naked attribute won’t prevent the inclusion of the termination code.
BSTR strings are COM based strings. They are allocated from COM-memory and consequently the memory can be managed by both the provider of the COM-object provider and the client. That is the first difference. Next a BSTR contains UTF-16 coded wide characters, which I discussed in ANSI and UNICODE. The way COM stores a BSTR is much the same as GB stores a String variable. In fact, a BSTR is 32-bits location that stores a pointer to dynamically allocated memory with UNICODE formatted characters. The length of the BSTR is stored In front of the BSTR, again like GB’s String data type.

Use Variant for BSTR
Although, GB provides hidden support for BSTRs, the only way to get access to a BSTR is by using a Variant. The following example assigns a GB-String to a Variant. At runtime the code allocates a BSTR by calling SysAllocStringLen(0, Len(GB-String)) followed by copying the converted GB-String to the returned address. The address of the BSTR together with its data type is stored in the Variant. When the Variant variable goes out of scope, the BSTR from the Variant is released through a call to SysFreeString(address).
Dim vnt1 = "Hello"
Now it gets interesting. After GB invoked the SysAllocStringLen() COM API, it converts the ANSI string to UNICODE using a private conversion routine interspersing zero’s between the characters see ANSI and UNICODE. GB does not turn to the MultiByte*() APIs Windows provides, because GB supports ANSI characters only. In the conversion process to UNICODE no characters will be lost and the private function is extremely fast.
An optimized UNICODE conversion function
This knowledge makes it possible to obtain a UNICODE-string (not a BSTR) from a String argument through our own optimized conversion routine. Note
  • A UNICODE string is required if you want to use the Wide version APIs.
  • A UNICODE string does not have a length field in front of it. It is not a BSTR. It only specifies how much bytes a character occupies (2).
  • It’s memory is managed by the program through malloc() – no COM memory - and it ends with two null-bytes (although it seems 1 is ok as well).
  • The converted ANSI argument is placed in a String only because it is a convenient data type to store consecutive data.
The function makes use of the BSTR allocation and conversion functionality of the Variant.
(The $Export is there because it comes from a .lg32 file).
Function WStr(vnt As Variant) As String Naked ' Return UNICODEd string
  $Export Function WStr "(AnsiString) As String-UNICODE Naked"
  Dim BSTR As Register Long
  BSTR = {V:vnt + 8} ' BSTR address at offset 8
  Return StrPeek(BSTR, {BSTR - 4}) ' <- 28-06-2017="" font="" updated="">
1. A function very well suited for the Naked attribute, because it does not contain local variables that contain dynamically allocated memory that would otherwise require explicit release code.
2. The argument of the function is ByVal As Variant. This forces the caller (calling code) to create a Variant and than pass it by value by pushing 16-bytes (4 DWords) on the stack. Whether the Variant is passed by value or by reference, the calling subroutine is responsible for freeing the BSTR stored in the Variant. However, ByVal is interesting because …
3. The GFABASIC-compiler provides a hidden optimization when you pass a literal string to a ByVal As Variant. A ByVal Variant requires16 bytes to push on the stack, but the UNICODE characters the Variant points to are already converted at compile time. Therefor the following call is extremely efficient:
Dim t$ = WStr("GFABASIC")
The GFA-BASIC compiler stores the literal string “GFABASIC” as a UNICODE sequence of bytes (2 per character) and does not need to allocate (COM) memory and convert at runtime. This also relieves the caller from releasing the BSTR-COM-memory, so the calling function doesn’t need to execute Variant destruction code.
Assigning a UNICODE formatted string this way, is almost as efficient as initializing a String with an ANSI literal string. It only takes a few cycles to call and execute the WStr() function.
4. The caller provides the String variable to store the return value of the function. That is the function’s ‘local variable’ WStr is silently declared in the calling subroutine. The hidden string is passed as a ByRef variable to the function. The return value (String) is directly assigned to the hidden variable. If an exception would occur in function Wstr() the termination code of the caller will release the hidden WStr string variable. (Therefor Naked is perfect for this function: it doesnot need to provide explicit release code.)
5. Inside the function you can see two more optimizations. First the local Long variable that stores the address of BSTR is a register variable; no stack memory and copying required. The other optimization is the Shl 1 expression that multiplies the length of the BSTR by 2. This results in an integer asm add eax, eax instruction, rather than a floating point multiplication. Also a significant optimization.
6. Other mathematic operations like V:vnt+8 and BSTR-4 are relative address operations and are properly compiled into indirect addressing instructions. So, no chance here to optimize.
I went in some detail to explain the function hoping you’ll find it useful. I hope to tell more about the way the compiler constructs subroutines and performs optimizations.