User:Msw/Testbed

The AS/400 system, taken as a whole, (considering both the hardware and the software that comprises the system), has many unique features. The following are some of the main ones:

• High Level Machine

    (NOTE: need to provide summary details for each category)

• Object Based
• Single Level Storage
• Hierarchy of Microprocessors
• Operating System: OS/400 (aka. IBM i)

These are nicely(?) explained in the (/long-winded/) video linked below.

Also, we will highlight some features that are unique, as compared with other more common platforms, and not mentioned above.

Layered Architecture

CPF and XPF were originally designed and implemented using a layered architecture.

At the lowest level is the actual hardware. Firmware (microcode) creates the ISA or instruction set architecture as seen by the developers of the first layer above the hardware, known as the LIC or Licensed Internal Code. This is roughly the equivalent of a "kernel" in modern OS terms, but it is closer to a "micro-kernel" a la the MACH micro-kernel project. In fact, IBM participated in the MACH research project with Carneghie-Mellon University (CMU), and IBM Rochester Labs was influenced by some of its design features.

For the System/38 and early AS/400 models, they were microcoded CISC machines, and the architecture implemented was called the Internal MicroProgramming Interface (IMPI). IBM used a proprietary language called PL/MP, similar to PL/S on the mainframe, for developing this LIC layer written in PL/MP.

NOTE: what follows needs work -- I do not like the "split in half" characterization; it is way too simplistic and does not accurately reflect the actual system.

The "kernel" is called Licensed Internal Code (LIC) and handles hardware dependent stuff. This includes features like multiple processors with dedicated memory just for I/O tasks.

NOTE: this is grossly mis-characterized. It was hardly only borrowed from the mainframe. By then, the IBM System/34 (and later System/36) both had "channels" and separate I/O processors, and the System/34 pre-dated the System/38 by several years. Even the System/3 and System/32 had a sort of "channel" -- in effect a DMA access to main memory that could be accessed by various I/O processors in parallel with the main CPU.

Above the MI virtual machine created by the Licensed Internal Code (LIC) is OS/400 and its components that System Administrators and Users usually see and work with. IBM calls this the Extended Control Program Facility (or XPF), that evolved directly from the IBM System/38 CPF, the immediate predecessor of the AS/400.

The Machine Interface (MI) OS/400 compilers do not output the actual CPU native binary instruction set (ISA) directly. That would defeat the whole idea of the MI "virtual machine" and the architecture of single-level storage that hides many of the actual details of the physical hardware. MI assembler language instructions, and their binary form, may be thought of as a kind of intermediate code, similar to Java ByteCode. This makes program objects larger than expected, but this "binary creation template" that houses the MI binary program creation template, is what enables unprecedented upward compatibility.

Program objects restored to a newer machine with different hardware, processor architecture, DASD technology, etc., can just be started like on the old platform. The start will take a few moments in that the SLIC recognizes missing CPU native code, compiles the intermediate byte code and adds the result to the program object. After that, the program is started and runs as native code on a possibly completely different CPU architecture.

IBM exploited this possibility when they switched the hardware from a custom 48-Bit CISC IMPI CPU architecture to the PowerPC AS architecture in the mid-1990s. In later incarnations, there have been some restrictions regarding this feature, requiring applications compiled with obervability turned on.

Backwards Compatibility

IBM has a long history building platforms that span a multi-decades of upward compatibility at binary level. The AS/400 is no exception. Programs compiled on earlier platforms and/or versions can be transferred to a newer machine and usually run without hassle.
Source code written with older releases of OS/400 usually compiles without any changes on newer platforms.

These features may not seem to be of much interest, at first glance. But, this enables developing an application on an older system and then running it on any newer AS/400e, iSeries, System i or IBM i system. This could be seen as a base requirement for more free and Open Source Software: It is easier to acquire an older machine.

Unfortunately, transferring objects from newer systems to older versions is not quite as easy. The Savlib command writes stuff into a pre-created Save-File and records the release string of OS/400 where it was run. It is possible to set pre-set different values but the newer the utilized OS version, the newer the oldest possible compatibility level. V4R5 permits V3R2 as oldest release and select V4 releases.
Transfers from old to new systems are painless, though.

Single Level Storage

Since it's first incarnations the AS/400 creates a virtual 128 Bit address space for the upper OS layer and application programs. This comprises physical memory and hard disk space.

A hard disk may be seen as a giant memory area mmap'ed into the address space of upper layer OS components. Swapping and Paging isn't done on the application layer as most conventional systems do, but on the address layer. Applications simply request contents from addresses and the layer below the OS called SLIC takes care of shuffling the data into RAM, should a page fault occur. Thus, page faults can occur also for data, not only for executables.

This concept also explains why the AS/400 platform lacks a conventional File System on it's disks. Since disks are essentially memory mapped into virtual memory space, everything visible is just an object at an address with certain properties. This allows to easily share these objects between requesting entities.

The advantage of this approach is the ability to do very fast context switches (if no or only a few page faults occur). This is the reason why even older machines with 100 % loaded CPU still have an astonishing fast response time when compared with common platforms of that era. If disk accesses are necessary for a context switch to take place, there's no notable speed difference to common platforms, though.

Disk Allocations

The usual way the system allocates space on secondary storage (aka: hard disks) is to spread objects over as many disks as possible. In modern RAID terminology, this is referred to as spindles (from the stack of rotating platters' axis), while IBM AS/400 documentation refers to the same concept as disk arms: The actuator moving the read/write heads over the desired cylinder — a set of tracks — on the rotating platters.

This spreading of data has the same advantages and disadvantages as RAID 0:

• with increasing count of disk arms, the chance that a given data block is located on an idle drive increases and thus decreases latency through exploiting parallelism: the more applications run, the more data can be picked up from individual disks, the more overall processing work can be done.
• since data is located in the system just once, the failure of (at least) one disk has hard to predict ramifications. Objects become damaged because parts of them are no longer accessible.

From early on, the concept of a dedicated I/O processor allowed for easy implementation of data mirroring, to compensate for data loss because of a malfunctioning disk drive through redundancy.

Later, the well-known RAID-Levels have been made available through the availability of dedicated storage controllers.

Memory Pools

The described concept of the platform relying on paging as primary means to bring in data and code into the faster main memory for actual processing has some implications on main memory constrained machines. Once, RAM was extraordinarily expensive. Without precautions, it was easily possible to start many programs at the same time, each one competing for presence in RAM. When seriously overwhelmed with concurrent requests, this situation leads to Thrashing, a situation where the machine is completely busy with paging activity, but does no useful work anymore because of competing tasks.

OS/400 introduced the concept of memory pools for that purpose. A memory pool mainly has these attributes:

• Size,
• Shared or Private,
• Maximum Activity Level.

Pool can be more or less translated to RAM Cache: What's already in the pool's memory is cached, because it mustn't be handled by the paging logic.

Pool Size

The Size of a memory pool is just how much RAM is set aside for applications being directed to run in this pool. More RAM means more cache and less disk activity for shoveling work into RAM.

Shared or Private Pool

Shared pools allow more than one group of applications to utilize a memory pool, while private pools are reserved for just one group. See Subsystems below for the explanation of this grouping. This brings in another consideration: The more pools one creates, the more unused memory is left over from excess memory partitioning.

Maximum Activity Level

The Maximum Activity Level of a memory pool is the overall count of applications which are eligible to actively run in a given pool in parallel. This eligible count only considers applications which require actual access to the CPU, but not sleeping ones, waiting for some application-external event. This setting reduces competition for memory amongst applications within that pool. But at the same time — to a lesser extent — overall competition for main memory, system-wide.

Pool Summary

It is important to consider that this competition for RAM is not limited to actual code being in main memory, but also for data. Heavy paging can be quite normal for a batch job reading through a huge database file, and only a minor brake on other applications running. As long as the application code stays in main memory, that application can run again as soon as more data arrives in main memory to be processed. Because batch applications (ideally) use a different memory pool than interactive applications, there is no competition for main memory for these two application classes being confined in their respective pools.

Many Books have been written about performance tuning the AS/400 system. As you can see, there are many aspects to consider which values of these settings lead to the sweet spot of a machine doing maximum (batch) work while minimally affecting interactive users. Dynamic scenarios have been described where interactive users are not considered at non-office hours, giving batch jobs more memory and thus allowing faster processing.

In current reality, there are two groups of users, and both of them rarely need to consider the points outlined above:

• Hobbyists are most often the sole users of a system. Thus, overall system activity levels are low — most often, the machine idles.
• Professional users with current systems have RAM in abundance. In addition, the increasing ubiquity of solid state storage with its intrinsic lack of mechanical positioning latency boosts I/O throughput.

For both groups, the directions given in the Post-Install Optimizations article gives an almost perfect starting point for the self-adjustment of the system itself to balance into a good partitioning of RAM into existing shared pools.

Expert Cache

In short, this is a setting of a pool where the system itself decides the block size of data being read from disk and put into memory. Without this, transferred blocks are of a fixed size, thus increasing processing overhead. CHGSHRPOOL PAGING(*CALC) switches Expert Cache on for a given pool. On the other hand, smaller data transfers decrease the wait time until some other disk access can be satisfied.
Usually the recommended setting is enabled. Allow the system to adjust itself. Note: The default setting is disabled.

Subsystems

A subsystem is a way to group application programs. Subsystems are defined by a subsystem definition object. This object allows for many attributes to be set, to prepare the desired run-time environment. See Subsystems for a more in-depth illustration about the underlying concept.

Running Application Programs: Activation and Activation Groups

Since programs are already existent as objects in the single level store memory space, there's no need to load them from disk as common OS' need to do. To actually run such an application program needs some housekeeping, though. Commonly a program needs a Stack and a Heap for variables and other internal data structures to be able to run. The AS/400 is no different in this case, although this stuff is buried very deep into the LIC's innards.

Actually starting a program to be run on an AS/400 is called activation. This task creates necessary storage as outlined above and adds the program to to the job scheduling queue. This is also necessary to provide a properly isolated environment when multiple users start the same application.

Completely different compared to common platforms, it's possible to share some of this space between programs. This shared space is called an activation group. That is, while creating a new program (by compiling and linking), one can tell the compiler/linker if the program is to be run in

• the default activation group, that contains most of the OS,
• the caller's activation group, to allow a more dynamic approach, or
• a new activation group that is to be created for every instance the program has,
• a common activation group with a programmer specified name.

As long as any program of any activation group is activated, so is the activation group. Data structures will be destroyed only when the last program terminates, or the job ends, or when a group is explicitly deallocated with the RCLACTGRP program. For details see weblinks.

Sharing an activation group shares (amongst other data):

• Static variables,
• Open files and file pointer positions,
• and more…

There's a certain intersection of functionality between a common activation group and fork()ing an application on Linux/Unix platforms. Even if it is not exactly the same but maybe it gives you an idea.
On OS/400, programs to share an activation group must exist as separate objects while on Linux/Unix all code is either linked to a monolithic program or uses additional code with the aid of shared libraries.

Standardized UI

The whole UI revolves mainly around forms and menus, accompanied by static text. The layouts of these screens are somewhat standardized. AS/400 screens are very recognizable because of this.

See also: About Green Screens and mouse-clickable UIs.

Predictable Command Naming

For experienced users who won't repeatedly navigate though nested menus, commands may be entered on the command line instead of menu numbers. These commands consist of two or more parts, built of abbreviated english words. Abbreviations often take place by omitting vowels and shortening the result to less or equal four characters. The first one being a verb and the second one a subject, optionally followed by more building blocks.

A few examples may illustrate the previous statements:

wrksplf

Work with Spooled Files

chgmsgq

Change Message Queue Properties

crtdspf

Create Display File

dltf

Delete File

dspmsg

Display Messages (from Message Queue)

See also IBM i Control Language and CL Tricks.

Outstanding Reliability

Software is a complex thing built by humans. Humans make errors. Thus, Software contains errors. This is an inevitable fact.

IBM managed to make OS/400 not bug free but caught most problems that lead to a complete crash of the operating system. There may be edge cases that are usually hard to spot.

Also, IBM built multiple measures against defective sectors on hard disks being catastrophic into the OS from early on. Since there is no classical file system, most often, read errors only affect one or more objects that can be easily restored from backup.

Reliability is not the same as availability. Some tasks can only be done when OS/400 is running in a restricted state. This includes but is not limited to a full system backup or certain hardware failure recovery procedures.

Rapid Development

OS/400 comes with a set of tools and supportive facilities within the OS to ease development of application programs. Usually, development comprises of:

• Definition and creation of database tables, display forms and printer output,
• Writing code that references these files and shovels data just by issuing proper READ and WRITE (among other) requests to these files.

The mentioned files are typed in a format called Data Definition Specifications or DDS. It provides data field definitions, field definition references between files, arbitrary placement of static text (not for database files) and a multitude of flags for content to be shaped to fit for the desired presentation.
The Screen Design Agent program (strsda) provides a pseudo-WYSIWYG-interface for creating screen forms and menus.
The Report Layout Utility program (strrlu) provides a pseudo-WYSIWYG-interface for creating layouts for printer output files.

Actual programming can be done in RPG, COBOL, and C/C++. Output of the compilers (called modules) created by any of these languages can be linked together to create an actual program object or a shared library (service program). Any language may call functions of any other language as long as parameters have been defined to be compatible.

The compilable CL "shell"-script language and REXX-Interpreter help in easing traditional automation tasks on the command line level.

Programming is done within the Program Development Manager collection of programs (strpdm). The most used component is the Work with File Members application, wrkmbrpdm, to list source code physical file content. This in turn calls the Source Entry Utility (SEU) with the appropriate parameters for actual code writing/editing with the desired existing or new file member.

SEU provides a linetype-aware form for RPG and DDS, easing the matching of the appropriate columnns. It can be invoked as usual: With F4.

See also: How to program an Application

Times they are a changin'

Today, development tasks are most often done on Windows PCs, with IBM provided tools, that pull a local copy of a source file to edit.

Also, since the late 1990's, IBM has been trying to establish Java as the language of choice, along with helper programs for shifting away from the old fashioned character based applications (green screen) to a pure web-browser based way of user's interaction with the system.

Also, the AS/400 Language, RPG, has been enhanced and modernised slowly but constantly. While most examples in this collection of knowledge are based on ILE RPG IV, newer OS releases were accompanied by compilers allowing to escape the punched card appeal of RPG more and more. V5 introduced the possibility to write actual code (not definition statements, though) in a free form, while with some V7 all parts of a RPG program may be written without being forced to write statements at a certain position within a line. This generates a load of syntactical changes to be learned.

Examples of positional code are slowly vanishing from Internet Websites. Looking up how to do stuff in positional code is getting increasingly difficult. Asking around in mailing lists for positional code examples sometimes yield unhelpful and cynic comments.
Still, many companies have old to very old code to be maintained that predates compilers allowing free syntax. Providing mainly positional RPG examples is supposed to help understanding this old way of writing RPG, while examples for newer techniques are easily found with an internet search engine of choice.

Weblinks

• IBM AS/400 The Best Architecture In The World 1992, Video at YouTube.
• Introduction to ILE RPG Activation Groups, McPress online.
• i5/OS V6R1: Upgrade Now or Later? with some background information regarding code observability