SQL

SQL is an acronym for Structured Query Language.

My only experience in programming is Omnis. I started with Omnis Classic using Omnis DML, that is using the Omnis date file with file formats... no SQL)

Herb Payerl, the person who introduced me to Omnis, advised me that SQL was the way to go, so when I bought Omnis Studio, I decided to switch to using SQL as well. I had zero experience in SQL. Learning Studio and SQL at the same time just about did me in. There were times I thought I was making two big mistakes; Omnis Studio, and SQL. But everywhere I listened and read, SQL was the way to go, so I kept at it. Once I got the hang of using SQL I learned to love it. Now I'm a convert ... SQL is the way to go, and Omnis Studio works wonderfully with it!

SQL vs DML

Opinions abound on this topic and everyone is entitled to their opinion. I used the Omnis DML (Omnis data file with File Classes) for many years and then reluctantly switched to using SQL to access the Omnis data file. Once I got through the SQL learning curve The following are some reasons why I believe SQL is the way to go.

1. Omnis DML is not conducive to object-oriented programming (OOP). With Object-Oriented Programming you don't want the data inside your objects to be affected by the actions of other objects and methods. The current record buffer (CRB) used by DML is global which can be a headache for object-oriented programming.
2. With Omnis DML you always had to be conscious of the CRB and be sure you have all the connected records in the CRB when you do updates (or your connections get messed up). In SQL there is no CRB, no prepare for update, and no connections going on in the background.
3. With Omnis DML if you deleted a field from a file class, these #??? would show up all over the place. Quite a job to clean up. In DML if you want to added a new field while you are coding, you must first go to the File Class, enter it, then continue with your code. In SQL you can write the code with the new fields, and then go and add them to the file.
4. Omnis DML is usually written modal. You have an Enter Data state and the user must OK or Cancel. When I switched to Omnis Studio, going to modeless data entry made it a lot easier to code.
5. Omnis DML only works with the Omnis data files. With SQL you can move your application to high powered SQL databases.
6. Omnis DML does not support commit and rollback transactions, which is very important for maintaining data integrity in mission critical applications.
7. Omnis DML does not support or enforce foreign key relationships. You are responsible for the integrity of foreign keys (making sure the parent record exists) and enforcing any constraints (prevent deletion of a parent record if their are child records linked to it)
   
   
8. Omnis DML does not support multiple joins to the same table.

Omnis Data File vs. RDBMS

For the first few years of my journey into the land of SQL I used the OmnisSQL DAM with the Omnis data file. The OmnisSQL/Omnis data file was free. All the other RDBMS (Relational Database Management Systems) that had Omnis Studio DAMs had to be purchased. That is no longer the case. Omnis Studio developed a DAM for MySQL which is a free RDBMS, and FrontBase changed their pricing structure so that their RDBMS is now free. If you want FrontBase tech support you need to purchase that separately.

Now that there are free RDBMS solutions available, why would you bother using the Omnis data file?

There are a couple of reasons for using the Omnis data file:

1. The base developer version of Omnis Studio only allows connection to the Omnis data file. You have to purchase the Enterprise edition to connect to other RDBMSs.
2. For demos which you ship to potential customers the Omnis data file does not require the installation of any additional software.

One of the problems I ran into with using the Omnis data file for demos is if you have any complex table relationships. Selecting and fetching records can be brutally slow with the Omnis data file for complex table relationships. I had one demo which was embarassingly slow with the Omnis data file. I spent hours trying to tweak the select and fetch to speed it up and had very limited success. I moved the data over to FrontBase and the demo changed from being embarrassingly slow to impressively fast!

Another problem with using the Omnis data file for demos is that you can't use views. Views are extremely handy for doing multiple joins to the same table. For example, if you want to join a correspondence record to the contacts table several times. (From, To, Cc) you can't do this with the Omnis data file. The only way to to write a work around which pulls records and joins them manually in Omnis Studio code... a lot of extra work for the programmer, and much slower performance.

Learning SQL

SQL was something I had to learn when moving from Omnis Classic to Omnis Studio. After reading a number of books on SQL I purchased Mastering SQL by Martin Gruber. This is the reference book which I turn to when I need to figure out a SQL script. Mastering SQL has depth, but doesn't assume too much knowledge on the part of the reader. A nice balance for programmers who are new to SQL.

Bind Variables

Bind variables are very important to use in your SQL script related Omnis Studio code.

Bind variables allow you to put variables in the SQL script that gets passed through to the database. There are several advantages to using bind variables:

1. Avoid mismatched data type SQL errors.
2. Solves problems with single and double quotes in the comparison string.
3. Ensures that date, time, and timestamp values are in the right format for the RDBMS.

To use bind variables, you simply enclose the variable you want to used in the SQL script with @[].

The following example uses bind variables for the date range.

Calculate DateFrom as fday(kMonth,#D)
Calculate DateTo as lday(kMonth,#D)

Calculate SQLText as "SELECT * FROM Salesorder"
Calculate SQLText as con(SQLText," WHERE OrderDate >= @[DateFrom] AND OrderDate <= @[DateTo]")

Do StmntObj.$execdirect(SQLText) Returns FlagOK

One thing you have to be careful with bind variables and table classes is that the bind variable must be visible to the table class instance. That means inside the table class you must use instance variables for any bind variables. If you pass in comparison variables as parameters, you must copy them to instance variable before using them as bind variables.

Use the columns of a row variable for your bind variable values and pass the row in as a parameter to your table class method. The table class method can then copy the parameter to an instance row variable. The following sample code demonstrates the technique which we use in StudioWorks.

; Some method which gets a batch of order by date range.

Do List.$definefromsqlclass('tSalesorder')
Do List.$sessionobject.$assign(sessobj)

Calculate DateFrom as fday(kMonth,#D)
Calculate DateTo as lday(kMonth,#D)

; The SQL text specifies column numbers for the bind variables.
Calculate SQLText as "WHERE OrderDate >= @[2] AND OrderDate <= @[3]"

; Use columns of a row variable for the variables.
Calculate WhereRow as row(SQLText,DateFrom,DateTo)

; Call a custom $getWhere table class method.
Do List.$getWhere(WhereRow)

; $getWhere (table class method)

; Copy the where row to an ivar row.
Calculate iBindRow as pWhereRow

; Get the SQL text from column 1.
Calculate SQLText as pWhereRow.C1

; Loop through columns 2 to the end.
For ColNum from 2 to pWhereRow.$colcount step 1
   
   ; Replace the column number @[#] with @[iBindRow.C#]
   Calculate Find as con('@[',ColNum,']')
   Calculate Replace as con('@[iBindRow.C',ColNum,']')
   Calculate SQLText as replaceall(SQLText,Find,Replace)
   
End For

; Execute the select with the SQL text that now points to the bind variable.
Do $cinst.$select(SQLText) Returns FlagOK
If FlagOK
   
   Do $cinst.$fetch(kFetchAll) Returns FetchStatus
   Calculate FlagOK as FetchStatus<>kFetchOk
   
End If
Quit method FlagOK

Single Thread vs. Multi-thread DAMs

Earlier versions of Omnis Studio used single threaded DAMs (Data Access Modules).

When Omnis Studio version 3 was released it included multi-threaded DAMs to support the Omnis web server. The multi-threaded DAMs were nicknamed V3 DAMs and the single-threaded DAMs were nicknamed V2 DAMs.

Maintaining the V2 and V3 DAMs was an extra load on engineering so eventually Omnis announced that the single-threaded V2 DAMs would no longer be updated.

The single-threaded DAMs have been phased out, so you shouldn't even bother with them, unless there is a specific reason you must support them in your application.

Multi-threaded DAMs

The multi-threaded DAMs are important to the Omnis web clients. With the web client setup, a single instance of an Omnis application is running on a server waiting for requests. Multiple web clients can be sending requests over the internet. In a single threaded architecture if webclientA requests a report that takes a minutes to produce, the other clients would have to wait in the queue until the webclientA's report was completed. For the interent that is not acceptable. The multi-threading allows your Omnis application to start serving webclientA, jump to webclientB start serving them, jump to webclientC, back to webclientA, etc.

The multi-threaded DAMs also bring object-oriented style architecture to the DAMs. With the single-thread DAMs the Set current session Omnis command is global, so if you were running multiple sessions or multiple applications at the same time you had to be very careful with your code to make sure the correct session was current.

Sessions and Cursors

Starting with zero SQL experience, the concept of sessions and cursors was very confusing to me.

Here's my current understanding of sessions and cursors.

• Sessions - When you logon to a database you begin a session with the database. Think of the session like a telephone cable that connects your application to the database. Once the cable (session) is connected you can now have two way communication with the database.
  
  You can start multiple sessions with the same database and/or multiple databases. Each cable (session) is independent and can have different settings.
  
  One session could be set to automatically commit every transaction, and another session could be set to manual commit, meaning the application could do a batch of updates, and then send a commit or rollback message to the database server to either commit all the changes or cancel them.
  
  An example of this would be having an autocommit session for getting the next primary key from a table which stores the last used primary key for each table, and a manual commit session for the rest of the database communications.
• Cursors - I struggled for quite a while trying to understand cursors, only to find out that I really didn't need to worry about them. (At least for most applications.)
  
  When you do a $select to a SQL backend server, the server prepares a list of the records you've selected. Then with a $fetch you can grab small chunks of the select list records from the server.
  
  If you declare another cursor and do $select and $fetch on another table (or maybe even the same table), the server now has both your select lists available to you.
  
  You can now jump back and forth and $fetch records from either select list using the cursors. If you do another select with the same cursor, the old select list is replaced with the new one.

Transactions & Concurrency

If you are new to SQL, probably one of the most confusing things when you are getting started with a real RDBMS is figuring out this business of transactions and concurrency. For many of us it would be nice if there was just one way to handle transactions and we would just follow that way. But it isn't that simple. There are many different choices, and you need to work through learning and understanding transactions and concurrency in order to figure out what will work best for your situation.

Many people learn in North America learn to drive in a car with an automatic transmission. Then one day they step into a car with a manual transmission. They have more control with the addition of a clutch and stick shift, but they need help figuring out how to use them. Much of the explanations that abound for transactions and concurrency assume we already know how to drive the SQL car. They explain how they work but few give simple examples, and suggest strategies, and recommendations for implementation. Hopefully this section helps you with learning to drive the SQL transactions and concurrency car. :-)

• Transactions are groups of SQL statements that are effectively executed as single units.
• Concurrency refers to mechanisms that the DBMS uses to keep operations on the same data by simultaneous users from interfering with one another. (Think of concurrent users.)

You begin a transaction whenever you initiate a session with a SQL database. All the statements you enter will be part of the same transaction until you complete it by entering either a COMMIT or a ROLLBACK statement.

OmnisSQL does not support COMMIT and ROLLBACK, so in OmnisSQL every SQL statement is automatically committed and there is no ROLLBACK option. You can set any RDBMS to autocommit transactions.

With the multi-threaded DAMs you control the transaction setting with the $transactionmode property.

Do $sessions.SESSIONNAME.$transactionmode.$assign(kSessionTranAutomatic)

With databases that support COMMIT and ROLLBACK you should make use of transactions and set the $transacation mode property to kSessionTranManual when you are doing a batch or series of database updates.

Do $sessions.SESSIONNAME.$transactionmode.$assign(kSessionTranManual)

That seems simple enough, but unfortunately it gets much more complicated than that. (Although the SQL gurus would replace "unfortunately" with "fortunately".)

Depending on your RDBMS there are numerous options to the manual transaction mode. (The information that follows is gleaned from a FrontBase list server posting by Geert Clemmensen of FrontBase.)

The SQL syntax for setting the transaction options is as follows:

SET TRANSACTION [ISOLATION LEVEL < level >] [,LOCKING < locking >] [,< access >]

Each set of < > angle brackets in the above SET TRANSACTION statement has the following options.

<level> can be VERSIONED or SERIALIZABLE or REPEATABLE READ or READ COMMITTED

< locking > can be OPTIMISTIC or PESSIMISTIC or DEFERRED

< access > can be READ ONLY or READ WRITE

That gives you a lot of combinations! (4 x 3 x 2 = 24 combinations) But life is simpler, experience tells us that only a few combinations really matter.

Note: VERSIONED is specific to FrontBase. It is a read only isolation level.

SET TRANSACTION ISOLATION LEVEL SERIALIZABLE, LOCKING PESSIMISTIC

• Good for short transactions (read and/or write).
• Required for table definition changes. (eg. ALTER TABLE)
• Has the smallest overhead for write transactions (on lock per table involved).

SET TRANSACTION ISOLATION LEVEL VERSIONED

• Very good for read only transactions.
• Can't take advantage of table caches.
• No locking overhead.
• The database "snapshot" can be shared by as many clients as needed.
• The locking discipline and access mode doesn't matter.

SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, LOCKING OPTIMISTIC
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, LOCKING PESSIMISTIC

• Maximum concurrency will occur (OPTIMISTIC setting).
• Row-level locking (the most "expensive")
• Good for longer transactions (read and write)
• Transaction conflicts will occur!

SET TRANSACTION ISOLATION LEVEL READ COMMITTED, LOCKING OPTIMISTIC

• This is like REPEATABLE READ, LOCKING OPTIMISTIC.
• Read only transactions will never yield a transaction conflict.

Editor Note: This setting is close to VERSIONED.

Everyone has their own opinions on what is right and wrong for transaction settings, so feel free to make your own choices. The following is what I decided to go with.

SET TRANSACTION ISOLATION LEVEL READ COMMITTED, LOCKING DEFERRRED, READ WRITE;

READ COMMITTED means that each time I select and fetch records, FrontBase automatically commits the transactions, so that the records are not blocked from other users who wish to read and write those records.

LOCKING DEFERRED means that if I read some records and then another user reads the same records, if I update my records, the other user is blocked from updating the same records. (Hope I got that right)

READ WRITE means that I can read and write records to the database, vs. READ ONLY.

For most of the work I do, I don't want to block users from reading the same records at the same time. Doing so, would cause unnecessary blocks and waits when two users want to look at the same record, or run reports on the same records.

I prefer to use an EditNum integer column in each of my SQL server tables. The EditNum is incremented +1 each time the record is updated. When the update is requested, I check to make sure the record's EditNum value matches the OldRow.EditNum value. This is accomplished by appending AND EditNum = OldRow.EditNum to the SQL update statement.

This methodology is considered optimistic locking as opposed to pessimistic locking which won't allow other users to access records which have been selected by another user and not yet committed.

The transaction settings are session specific. You can change the transaction settings for a session at any time during the session. (Although not for records you've already selected.)

To set the transaction settings you issue the SET TRANSACTION statement after the session has been started.

LOCKING DEFERRED (Explained by Geert B. Clemmensen of FrontBase)

The DEFERRED locking discipline is almost like PESSIMISTIC with the main difference being that when a row/table is referenced, a READ lock is acquired. If the row/table is updated, the READ lock is automatically updated to a WRITE lock. The DEFERRED locking discipline is thus ideal when updates occur less frequently than reads.

THE TWO SESSIONS STRATEGY

Another strategy is to use two sessions.

1. SessionA is used for doing selects for read-only lists and running reports. This session would be VERSIONED or READ COMMITTED, OPTIMISTIC, READ ONLY
2. SessionB is used for modifications and would be SERIALIZEABLE, PESSIMISTIC.

The only way to make transactions 100% secure is to SELECT and UPDATE the data in the same transaction using a PESSIMISTIC locking discipline.

With the 2 session strategy you might fetch a batch of records into a list that is presented to the user. When the use signals a request to edit a record in the list you refetch the record with SessionB and then write and commit it when the user signals to save the changes. SessionB users could could continue to read the record being edited.

Remember: You begin a transaction whenever you initiate a session with a SQL database. All the statements you enter will be part of the same transaction until you complete it by entering either a COMMIT or a ROLLBACK statement.

Note: The caveat with VERSIONED is that table caches aren't used during a VERSIONED transaction. VERSIONED represents history, table caches represent history being written. VERSIONED is in particular ideal for long transactions that only read data, e.g. report writing. READ COMMITTED, OPTIMISTIC, READ ONLY is recommended for connections that only read data.

StudioTips Documentation - Copyright 2006 Vencor Software