David Wimberly
University of Arkansas
October 2002
Revised January 29, 2003
The University of Arkansas provides employees with web access to personal and general administrative data. This data is managed by ADABAS on an IBM mainframe with application programs written in Natural. This paper presents four specific web enablement challenges which were encountered during this development effort, and our implemented solutions.
Discovering the techniques required to address these challenges was difficult and time consuming, especially for mainframe developers new to the web development world. We were shocked by the lack of how to information and instructional resources available. The intent of this paper and presentation is to save others time and frustration in addressing these or similar problems. We do not intend to imply that the solutions presented are the only or are the best way to address these issues. We do not consider ourselves experts in this field, but merely feel we have learned some lessons worth sharing -- many via the trial and error methods that seem to dominate the web world. We are pleased with our results and feel that our objectives for a web interface have been met.
This paper is technical in nature and will be of most interest to developers of dynamic web applications. Some knowledge of ADABAS, Natural, HTML, and JavaScript is assumed. The University is using the Com-Plete HTTP Server and its integrated Natural CGI interface for web enablement, which allows all programs to be written in Natural.
Our first challenge to web-enablement of our mainframe applications was to provide the same field help features our users were accustomed to and had become dependent upon in the 3270 environment. Basically, whenever a user did not know what to enter into a field he would press the PF1 help key while the cursor was on the field in question. This would produce a help window with a definition or explanation of the field and, where possible, a list of values from which to select. A selected value would be returned to the input field on the main screen. Figure 1 is an example of a mainframe green screen demonstrating this type of field help.
|
|
| Figure 1. Mainframe 3270 screen demonstrating field help. |
Providing these basic field help features on the mainframe has been standard operating procedure for the University for the past 15 years. Since the web is newer and offers a superior user interface, we had presumed that the techniques to provide the same or similar features would be documented and readily available. However, we quickly became frustrated in trying to find how to accomplish this seemingly simple task, both using web searches and our limited HTML documentation. We were also unsuccessful in finding examples on the web of other applications performing similar functions. Our local Java web development guru even told us it could not be done. It can be done, and is really not that complex. The solution relies upon JavaScript, Dynamic HTML (DHTML), and the Document Object Model (DOM).
For all fields we maintain a field definition on Predict.
This definition is accessed and displayed for our mainframe
field help.
For a field whose content is completely variable, such as a date or a street address,
this definition is all that is provided.
For a field with a small set of discrete values, such as M
or F for a person's gender,
the values are included within the Predict field definition.
By convention, this is accomplished by including the value to the
left of an equal sign within the definition, i.e. M = Male.
Our help routine parses the definition, finds these values,
and makes them selectable to the user.
A third type of help routine is developed for fields whose values are
defined on a file or table.
In this situation the user is allowed to search and scroll through
the table to find the appropriate value.
A template sub-program is provided as a model for the development of
these routines on the mainframe.
Similar field help solutions have been developed for the web. One CGI program provides help for fields which have only a field definition on Predict, another provides help when the definition includes values that may be selected, and a Natural ISPF macro is used to generate CGI programs that permit browsing and selecting values from a table. The following screen snapshots demonstrate these web based help facilities.
 |
| Figure 2. A main browser window with several input fields which have field help available. |
|
| Figure 3. Field help where only a field definition is available. |
|
| Figure 4. Field help where the definition includes a small set of selectable values. |
|
| Figure 5. The initial window, showing a field definition and search options, for field help where possible values are read from a table. |
|
| Figure 6. A subsequent window, with selectable values displayed along with paging and search options, for field help where possible values are read from a table. |
As previously mentioned, the key to this solution lies in the use of JavaScript for the necessary client control while relying on DHTML and the Document Object Model for access to the necessary components: windows, forms, and the fields of the forms. The solution will be presented in the following stages: opening the help window, passing parameters to the help CGI program, and returning the user's selection from the help window.
First, the user needs something to click on to direct the opening of the
help window.
This is done with a link (HTML anchor) where the link is the text of
the tag used for the input field.
A null link (href="#") is used because we do not want the
automatic link process to open the window, but will instead use
JavaScript in order to provide more control over the new window.
<a href="#" ... >Field Prompt</a>
|
Where the ellipsis appear, several additional anchor
attributes are specified.
The first is a target window name,
since we do not want the help window to replace the contents of the current
window.
Although "_blank" can be used to always create a new window, you can also
name the window and if a window by that name already exists
(the user previously asked for help but never closed the window)
it will be used.
target="helpWindow"
|
We also specify a tabindex value (supported by most
browsers) so that the cursor will not stop on
this link when the tab key is used to advance to the next input field.
(To get help, the user will need to use the mouse to point to and click
the link.)
tabindex="-1"
|
An additional convenience is to specify a title for the link.
In many browsers this title will pop-up when the user rolls the mouse
over the link.
title="Help information for ..."
|
JavaScript is used within the onClick event handler
to open the window (window.open method) and
cancel the normal action of the link (return false).
The JavaScript is enclosed by double quotes with the method call
separated from the return statement with a semi-colon.
The three parameters of the window.open method are
the url for the window (which will specify our CGI program),
the name of the window (which should match the name specified as the link target),
and a window feature list (a string of optional features separated by commas).
The return false cancels the link action of
the href so that it does not open an additional window nor
replace the window contents created by our CGI program.
|
The url parameter of the window.open method specifies the CGI
program that produces the content of the help window.
In our environment, this program needs to know the Predict field name for which
help has been requested and a two character application area.
These can be passed forward from the main page by adding them to the end
of the url as a query string.
In this way they become HTML form field values which can be read by the
CGI help program.
The beginning of the query string is denoted by a question mark,
fields are defined by name=value pairs, and multiple
fields are separated by an ampersand.
So, the first window.open parameter appears as follows with
a relative url (the CGI program resides in the current directory) and
values for fields pAA and pFld
(Predict application area and field).
onclick="window.open('uwohfdo?pAA=HR&pFld=EMP-LOGON-ID',
|
If the help routine permits the selection of a value to be returned to the
main window,
the help window must know the HTML form name and
field name that is to receive that value.
Using these the help routine, while executing in a separate window, can
alter the value in the main window thanks to the DOM.
(Note, to do this the CGI programs for both windows must be
executing from the same domain.)
The form and field names can also be passed to the CGI help program via the query string.
So we add them as additional
name=value pairs with standard parameter names
(we use wFm and wFld).
With this addition, the complete field help link appears as follows.
|
Within the help window, the possible values to return can be made selectable
using the same null link method which was used to open the window.
However, the onClick action in this circumstance must assign the
opening window-form-field the chosen value.
This is done by using the value property of the opening window's
form and field object specified in JavaScript as:
opener.document.[FORM].[FIELD].value=[VALUE]
|
opener is a property that allows reference to the original window
that was used to create the current window.
There we wish to access the value property of a field on a form of that document.
The [FORM] and [FIELD] are the values received by the
help routine as parameters while the [VALUE] is the selected value
to be assigned.
Needless to say, this JavaScript statement must be dynamically generated.
We generate it as part of a JavaScript function which also closes the help
window.
This function is invoked with the selected value as a parameter -- all via
the onClick associated with a null link.
The JavaScript function setVC for Privacy Code
which appears on the main window form webForm as
field EMP_PRIVACY_CD is:
|
Note: The field name EMP_PRIVACY_CD uses underscores
in the field name versus dashes which are used for our Predict field name.
This is because dashes are not valid within the DOM for naming objects
even though they are valid as an HTML form field name.
We dynamically translate the dashes in our field names to underscores so
that these JavaScript references will work.
The null link to invoke this function for a particular
Privacy Code value (1 in this example) is:
|
Once selected, the new value is immediately seen in the original window, form,
and field.
The window is then automatically closed (window.close()) since
it is no longer needed.
One alternative to opening a window for the selection of a value from a list is the use of the pull-down menu or drop-down box. This approach is widely used throughout the web and can be very effective. With this HTML input type, all possible values are immediately available for selection by the user. Although the pull-down menu generally works well, we choose not to use them for the following reasons:
We chose a browse-select user interface for our web applications. The intent is to minimize the need for users to key in record identifying data and maximize the advantages of the web for displaying information. Where possible this takes the form of a drill-down process where more detail information is exposed with further selections (mouse clicks, of course). This interface means we need to present information in a list format and allow selection of an item from the list. The list format presentation is easy using HTML tables. The challenge was how to select an entry and pass its key information on to the next program. Of course this has to be done entirely within the browser on the client side.
Our solution to this problem, as with our field help implementation, is based upon JavaScript and Dynamic HTML. The end result of the techniques used can be briefly seen via the following screen snapshots.
|
| Figure 7. A browse of Purchase Orders with an action selected to view invoices for the PO. |
|
| Figure 8. A browse of invoices for a selected PO. |
|
| Figure 9. A view of invoice data for a selected invoice. |
We use hidden form fields as the vehicle for passing the selected values forward to the next CGI program. The hidden fields receive their values via JavaScript assignments at the time of the user's selection. The selection is made using the null link concept employed for field help. Code fragments for each of these components will be presented and described.
Associated with each entry listed and available for selection, we will have one or more key fields which uniquely identify the displayed data. The values for these record identifiers are what we want to capture (based upon the specific entry selected by the user) and pass along to the next program for further operation. In the examples provided, it is first a Purchase Order Number and second an Accounts Payable ID (which is not even displayed in the list since its value is not relative to the user). These key values are assigned to hidden HTML form fields so that they may be read by the subsequent CGI program. The hidden fields must be embedded in the web page within the form definition. This definition for the PO number is:
<input type="hidden" name="PO_NO">
|
We pick a primary field (column) from the list and make it selectable by defining it as a link (HTML anchor). The user merely clicks this value to select the entry. The action to perform on the selected entry is specified at the top of the page via a radio button. Since we need no further information, once an entry is selected the form is submitted and the CGI program that will perform the specified action is invoked.
A null link (href="#") is used because we will control
what we want done via JavaScript.
In this situation, we will invoke a JavaScript function
(the same function for all entries)
via the onClick event handler
and pass a parameter identifying
the specific entry selected.
|
tabindex='1' is specified so that the cursor will stop on
these links when the tab key is used.
The title will be displayed when the mouse is placed on the
link.
setVS is the JavaScript function that will be executed when
they click the link, and 1 is the parameter that will be passed
indicating which entry was selected.
return false cancels the normal action performed by the link,
since our JavaScript code will do everything we want within the setVS
function.
The PO Number, 6001260, is the text we have chosen for the link.
The JavaScript function, setVS, invoked by the selection of
an entry is shown followed by an explanation of its components.
|
listEl1 representing
the PO Number), but there could be several key fields in other circumstances.
Each occurrence of the array is assigned a value corresponding to the same entry
listed within the HTML table.
The Document Object Model makes it possible for us to set the value of the
hidden field PO_NO within the HTML form webForm within
the current document (browser window).
The value assigned is from the newly created array based upon the occurrence as
specified via the function parameter le_x.
We also set another hidden form field, actionReq="Execute",
which will tell our CGI program that the user selected an entry.
We then use JavaScript to submit our webForm using the form's
submit() method.
Web development differs in many ways from terminal based connection oriented application development. The stateless nature of the web is one primary difference, though there are many ways to address this and to impose state onto a web environment. A larger issue is that the web browser can cache previously visited pages, the user is permitted to revisit those cached pages via the Back Button, and the user can initiate some action from a cached page which he fully expects to be successful. We impose state on the user so we know where and what he is doing within some process and then suddenly, the next time we hear from him, he is coming from a prior page asking to do something that was valid at a previous point in time. The caching of web pages is a powerful feature and one of many benefits of a web interface over the traditional 3270 terminal interface. However, this is an alien concept for a traditional mainframe developer -- not having the user continue from the page (screen) last generated.
Traditionally, the operating system maintains information about the program a user is executing (where he is) and within that program the developer maintains information about the data being operated on (what he is doing). This state information is the basis for the next operation that is performed (perhaps paging forward). In a terminal based environment only the information related to the current state, the last operation, needs to be retained. Doing so is a snap, its automatic. But in a web environment, we need to retain everything needed to continue any option available from any page previously visited. To do this, we need to maintain state data for each page. We could number the pages and store this state information off on a file in some generic format, to be retrieved later if the user submits a request from that prior page. Another option is to minimize and compact the state information needed and then bury it in the web page itself within hidden form fields. These values are then available to be read whenever an operation is re initiated from that prior page. In this way, the processing is no different whether the user interacts from the last page generated or from a previously visited page.
The techniques we have used to support the back button will be presented using our browse-select user interface. The following web page snapshots demonstrate this interface and how the back button may be utilized to continue browsing from a prior page, though only through a live demonstration can you see the true affect.
|
Figure 10. The browse of all services searching on keyword starting from
keyword 'vendor', with the selection highlighted for Invoices.
|
|
| Figure 11. The browse of Invoices with search criteria being specified for a vendor on or before a received date. |
|
| Figure 12. The browse of Invoices with search results displayed for the vendor and received date. |
|
Figure 13. The result of using the back button twice to revisit the
browse of all services from keyword 'vendor' and then
clicking the Page Back button.
|
In our browse-select user interface, the state information we maintain for each page includes the following data items.
The ISN of the first and last record of the file being browsed
is retained so we can browse back (using the first ISN)
or browse forward (using the last ISN) if requested.
The appropriate record is reread using a GET and the search key value is
established using the data from that record.
A READ or READ DESCENDING is then initiated
using the STARTING FROM ISN option.
It is a bit messier
when the search key includes a PE or MU field.
In this situation the occurrence of that
value in the first and last record is also retained.
To reduce the number of hidden fields and size of our HTML file, we
create one P10.3 field to hold the ISN and the occurrence
(stored in the decimal portion of the field).
We also have to use HISTOGRAM/READ loops with MU type descriptors and
HISTOGRAM/FIND loops with PE type descriptors.
(Note the restrictions for PE type descriptors discussed in the
Browsing ADABAS Data section later.)
These two hidden ISN/occurrence fields appear as:
|
We provide a list of menus or functions (programs) the user has visited from the main menu to the current selection. These are displayed below the primary page title within brackets with a > between each selection. These are active links and can be used by the user to jump directly back to any of those previous levels and reinitiate the last search specified. (Our menus are stored on a file and implemented as standard browse-select programs with fixed search criteria.) Home and Return links are also displayed at the top and bottom of each page and offer another way to go directly to the main menu or to return to the previous menu or function. To support these features we maintain what we refer to as a program stack with several pieces of information kept for each of these levels.
One piece of information is the Natural program name that was executed at that level to produce that page. This will be the program that is invoked if the user returns to that level. In addition to the program name, we tack onto the end of this the index of the search type that was used at that level (our PO browse offered seven ways to search the file, 1 through 7) and the index value for the Action that was selected. Both of these are one digit values (our own imposed limitation). They allow us to invoke the appropriate search when the user returns to that level and retain the action that was being performed. To reduce the number of hidden fields and the size of the HTML file, these are strung together into one HTML hidden field value with semi-colons used to delimit the levels.
|
In order to reinitiate the search at a prior level, we need the ISN (and occurrence if an MU or PE type descriptor is involved) of the first record that was displayed on that page -- which will also be the first record displayed when they return. In addition, we need to know whether the user was allowed to page back from that display (based upon the search keys that were initially specified). We retain that piece of boolean information by reversing the sign of the ISN value (we make it negative if they could page back). These first isn values for each level are also strung together using a semi-colon delimiter into one HTML hidden field, with a decimal portion representing an occurrence only constructed when necessary.
|
One other piece of state information we maintain per page is the title of the menu or program at each level. Technically this could be obtained from the data base, but we chose to carry it in hidden fields as well to avoid additional ADABAS calls.
|
With the above information contained on each page, we are able to support any of the actions the user may initiate from that page -- even though the page may have been resurrected from the browser's cache. This includes paging forward or backward from that page, returning to a lower level function, or making a selection from that page and going forward.
I am curious to learn how others have addressed the back button issues. Our initial efforts were to disable it, and only after meeting defeat on that front did we focus on making our browse-select work with browser cached pages.
In our browse-select interface, we wanted to permit the user to
page forward or back through data whenever possible.
Thanks to the newer ADABAS and Natural features of
READ...DESCENDING and
HISTOGRAM...DESCENDING, paging back is no longer a
problem (except for non-unique PE type descriptors, as explained in
PE Type Descriptors).
The main challenge is accomplishing this in the stateless web environment
where we may receive such a request from any page previously visited.
An additional challenge is establishing the proper Natural logic for
MU and PE type descriptors to ensure we continue with the correct record,
end with the correct record, and records are not repeated.
The description of our solution will start with general comments regarding how the starting points for the data access is established and how we determine when the paging operation will be permitted. Examples and discussion then follows for each type of ADABAS descriptor.
The back button discussion presented the concept
of retaining within each page as hidden form fields the ISN
(and MU or PE occurrence) of the first and last
record displayed.
When a page back or page forward request is received, the appropriate
ISN is used to re-access (GET) that first or last record.
Data values from that record are then used to establish the search values
for the READ or HISTOGRAM.
For an MU or PE type descriptor, the occurrence is also used to obtain the
correct value for the MU or PE field.
The direction for the read is established in a variable (A for ascending or
D for descending) and the READ or HISTOGRAM issued.
The READ will also include the STARTING WITH ISN
clause where the ISN obtained via the hidden form field is specified.
Since the read does not actually start with that value but with the next
higher (or lower) value, we do not re-read the same record we had previously
listed.
For an MU type descriptor, we follow the HISTOGRAM with
a READ which
includes the STARTING WITH ISN clause. One complexity here is
that we must only use the ISN from the hidden form field in the first READ
and for subsequent loops through the HISTOGRAM we must use low (zero) or high
(4294967295, 2**32 - 1) ISN values.
For a PE type descriptor, we follow the HISTOGRAM with
a FIND since a READ is not permitted.
Examples will follow for each descriptor type.
It may be beneficial to understand when paging options are permitted. In order for a page forward button to be available, we must have read one more record than we listed or the user has just paged back. In this way we know there is more data to be viewed. Page back is available if they have just paged forward, they were paging back and we read one more record than we listed, or if they have specified new search criteria and the values specified are not the lowest possible values for the search (such as a null date).
The PO search for a vendor on or after a date is a good example of a browse on a regular descriptor. This is a super-descriptor composed of vendor number and effective date. The vendor number must match, but we allow browsing forward and backward across dates. The search key definition is:
|
The GET and MOVE BY NAME statement used to restore
the search key prior to a paging
operation is:
|
The data base read statement and subsequent check for the same vendor is:
|
ADABAS does permit a read logical on an MU type descriptor.
However, unless the MU field is in the high order portion of a
super-descriptor where that value is fixed you will need to
do a HISTOGRAM first.
This is because with the READ you will not
know which MU field
value was used to provide the access to the record.
The HISTOGRAM will provide that value within the view.
Our Browse All Services includes an option
where you can browse the available services by keyword
for your given access group.
Keywords and access groups are both MU fields.
Since you cannot create
a super-descriptor from two MU fields, we created a third MU field
which we programmatically maintain with all combinations of values, and
this MU field is defined as a descriptor.
The user can browse the services forward and backward from any starting
keyword value (within a fixed access group).
( Note that this means that the same record can appear
within the list in multiple places -- once for each keyword associated with it.)
The definition of the search key is:
|
The additional view required for the HISTOGRAM is:
|
The GET and assignment statement (with the occurrence index value)
used to restore the search key prior to a paging
operation is:
|
The code to execute the HISTOGRAM to obtain the keyword values is:
|
For the first descriptor value obtained via the HISTOGRAM
we may be continuing to access records
from a previous page.
To bypass those records we use the STARTING FROM ISN
with the ISN we read from the hidden form fields and which we used to re-establish the
search key.
But for subsequent descriptor values
we want all records and therefore we set
the ISN variable to zero which will result in the STARTING WITH ISN
being ignored (whether reading ascending or descending).
|
Now we are ready to READ the records.
|
There remains only one more issue.
When reading by an MU type descriptor there is the possibility of reading the
same record two or more times.
(Note: this would not be an issue if you could do a FIND here instead of a
READ, however there is no FIND DESCENDING. See
PE Type Descriptors for more discussion on this issue..)
The check to see if the key word from the HISTOGRAM, HKEYWV.#MENU-KEYWORD,
exists within the the MU field of the record read, DBV.MENU-KEYWORD(*),
is not enough.
Envision the same record with keywords 'K1' and 'K2'.
We need to ensure that we have not read the record via the next descriptor
value, 'K2'.
The next loop through the HISTOGRAM will give us that value and we will initiate
another READ for it.
If the descriptor values were unique, you could read one record and stop.
But the same keyword may exist in multiple records.
So to keep from reporting the record twice (or actually three times),
we can keep track of the last ISN read and ensure that this current records ISN
is greater (or lesser if doing a READ DESCENDING).
So we have enhanced the IF...ESCAPE BOTTOM and added an
assignment to retain the ISN last read.
|
ADABAS does not permit a read logical on a PE type descriptor
and therefore a FIND must be used instead of a READ.
However, because there is no FIND...DESCENDING,
Natural and ADABAS cannot effectively support stateless backward
browsing with non-unique PE type descriptors.
The discussion and examples provided here show how to do this for unique
PE type descriptors using HISTOGRAM and FIND.
With only one record per descriptor value, a FIND(1) suffices and
you do not have to worry about restarting from within a set of records
that share a descriptor value.
This is not a problem going forward because FIND does support
STARTING FROM ISN, it just lacks a descending option.
Software AG has accepted enhancement proposal #958
to add this feature.
You can read more about this issue on Quest or Servline24.
I encourage everyone to let Software AG know what they think about this
proposal.
The example used for browsing by a unique PE type descriptor involves electronic transactions that are pending review by a desk. The reviewers of a transaction (identified by a desk) are stored within a PE group along with the review status, time, etc. The search key definition for this descriptor follows:
|
The HISTOGRAM view is:
|
The GET and assignment statement (with the occurrence index value)
used to restore the search key prior to a paging
operation is:
|
The code to execute the HISTOGRAM to obtain the key to the
pending transaction is:
|
Technically, the descriptor used in this example may not be unique even though we
are treating it as if it were.
If there were two records with the same value, we will only report
the one with the smaller ISN.
We are pretending it is unique so we can use the FIND (1) and
ignore issues that would arise if records for the same descriptor value
were needing to be displayed across web pages.
|
Questions, comments, or information requests can be made to David Wimberly at the University of Arkansas by sending e-mail to wdw@uark.edu or telephoning 479/575-6465.