Try-AS/400 - User contributions [en]

Porting C-Source to OS/400

2026-04-28T21:55:17Z

Greenfive: fixed typo

{{FIXME|Complete with Links to all necessary Books.}}
Because of the ''uncommon'' file handling, name length and character constraints, porting is already a mostly manual task.<ref>It's possible to use stream files in IFS as compiler input, but compiled objects have to be put into a ''QSYS.LIB'', library. So, the file name restrictions apply anyway. The line length problem is mostly gone, though.</ref>

Most often, the real challenge is to manually put together a ''config.h'' file. Since there's no capable UNIX shell environment, <code>./configure</code> is no option.<ref>With V4R5, IBM claims to have implemented the C90 standard in Compiler and C-Library. I did not yet find a readymade ''config.h'' with all statements needed for a C90 environment.</ref>

These are definitions for integers, being used often. This can be put e. g. into a separate header file:
<syntaxhighlight lang="c">
#ifndef __stdint_h
typedef unsigned long long int uint64_t;
typedef unsigned int uint32_t;
typedef unsigned short int uint16_t;
typedef unsigned char uint8_t;
#define __stdint_h 1
#endif
</syntaxhighlight>
<blockquote>'''Note:''' The IMPI (CISC) C compiler in V3R2 and earlier doesn't know <tt>long long</tt>, it's 32 bits only.</blockquote>

== Preparations for QSYS.LIB ==
Preparations are mostly done on UNIX/Linux side. This is required to make the older V4R5 compiler happy. Newer compilers might be more forgiving.

# Convert to native line endings for your platform, if the source files don't match.
# Convert Tabs to spaces:<ref>Tabs appear in SEU as only one blank. This makes source indentation hardly visible.</ref> <code>for FILE in *.c *.h; do cp ${FILE} ${FILE}~; expand -i -t4 < ${FILE}~ > ${FILE}; done</code>
# Find out longest line in all C- and H-Files. <code>wc -L *.c |sort -rn |head</code>
# Eliminate too long lines by wrapping. This is easily done with <code>set colorcolumn=80</code> in ''vim'', and scrolling through the files with a wide window. <ref>Wider than 80 chars, that is.</ref>
# Find C++-style comments and convert to proper C comments.
# Find function definitions which should be inlined, remove the word "inline" from the definition, and add a hint for the preprocessor/compiler that a function should be inlined before the function definition: <code>#pragma inline (''functionname'')</code>
# Find and remove text following <code>#endif</code> preprocessor statements in the same line (thought as comment).
# Create (at least) '''two''' appropriate SRCPF with matching maximum line lengths through the RCDLEN parameter for CRTSRCPF:
## One (or more) for C-Files (''may'' be named ''QCSRC''),<ref>It could be clever to create one PF per subdirectory.</ref>
## One for H-Files ('''must''' be named ''H'').
# Upload.
## If the file names are already shorter than 10 Characters (excluding file extension), just upload to a SRC PF: <code>for FILE in *.c; do echo -n "put ${FILE} MYLIB/QCSRC."; basename ${FILE} .c; done |ftp as400</code><ref>Automatic login and switching to ASCII mode is done though a ''~/.netrc''-file.</ref><ref>FTP ASCII takes care to convert ASCII to EBCDIC.</ref>.
## If the file names are longer, you'll have considerable fun to come up with shorter names. Possibly ''grep'' through, and replace names in all text files if you needed to shorten a ''.h'' name. Finally, put together a text file with appropriate ''put'' commands, and hope for the best.
# Change file type to ''C'': <code>for FILE in *.c; do echo "quote rcmd CHGPFM FILE(MYLIB/QCSRC) MBR($(basename ${FILE} .c)) SRCTYPE(C)"; done |ftp as400</code><ref>This is a good example how to exploit the ftp server's ability to call CL commands as a replacement for ''rcmd''.</ref> Note: You'll need to do this for header files, also!

The precompiler searches private includes in ''*LIBL/H'', without the ''.h''-Extension.

== Compile Cycle ==
Compile object files as follows:
CRTCMOD MODULE(MYLIB/MAIN) SRCFILE(MYLIB/QCSRC) DEFINE(DEF1 DEF2=1) OPTION(*LOGMSG) OUTPUT(*PRINT)
* Add <code>SYSIFCOPT(*IFSIO *IFS64IO)</code> to the compiler parameters if the code contains <code>fopen()</code> calls.
* If you added <code>#pragma inline (''functionname'')</code> to the code, add <code>INLINE(*ON)</code> to the compiler parameters

If compilation fails, output will go into a spooled file in the configured output queue. This is the most tedious task.
# Compile,
# check output,
# fix problems,
# <code>goto 1</code>.

If all object modules have been created, you can then link them to a ''*PGM'' object as follows:
CRTPGM PGM(MYLIB/MYPG) MODULE(MYLIB/*ALL) ENTMOD(*PGM) ACTGRP(*NEW) DETAIL(*BASIC)

== Footnotes ==
<references />

[[Category:Programming]]

Features that separate the AS/400 from common computer systems

2026-04-16T15:31:24Z

Greenfive: fixed typo + style

The AS/400 as a whole of Soft- and Hardware has a bunch of unique features as advertised by IBM:
* High Level Machine,
* Object Orientation,
* Single Level Storage,
* Hierarchy of Microprocessors,
* Operating System: OS/400.

These are nicely explained in the video linked [[#Weblinks|below]].

Reviewed in 2022, some are still relevant, some are no longer. Additionally, I'll highlight features that I think are unique compared to common platforms, and not listed above.

== Split OS ==
OS/400 may be seen as split in half.<ref>Not necessarily evenly sized halves.</ref> An good deal of it (about 1 GiB with V4) is called ''System Licensed Internal Code'' (SLIC) and handles hardware dependent stuff. This includes features like multiple processors with dedicated memory just for I/O tasks.<ref>This may be seen as a technical borrowing from the mainframe world within IBM. This technique is called ''Channel'' there.</ref>

Above the SLIC is the OS/400 and its components system administrators and users usually see and deal with. This is called [[wikipedia:Control Program Facility|CPF]]<ref>Control Program Facility</ref>, a leftover from the AS/400’s ancestor, the System/38.

== Byte Code ==
Compilers don't output CPU native byte code but kind of intermediate code, for example like Java does. This makes compiled programs surprisingly large in size but enables an unprecedented backwards compatibility on binary level.

Binary programs copied to a newer machine with different processor technology can just be started like on the old platform. The start will take a few moments in that the SLIC recognizes missing CPU native code, compiles the intermediate byte code and adds the result to the program object. After that, the program is started and runs as native code on a possibly completely different CPU architecture.

IBM exploited this possibility when they switched the hardware from a custom 48-Bit CISC CPU architecture<ref>Derived from the CPUs in IBM Mainframes of the 1980s era.</ref> to the PowerPC AS architecture in the mid-1990s. In later incarnations, there have been some restrictions regarding this feature, requiring applications compiled with ''obervability'' turned on.

== Backwards Compatibility ==
IBM has a long history building platforms that span a multi-decade backward compatibility at binary level. The AS/400 is no exception. Programs compiled on earlier platforms and/or versions can be transferred to a newer machine and usually run without hassle. 
Source code written with older releases of OS/400 usually compiles without any changes on newer platforms.

These features aren't of much interest on the first look, but they enable one to develop an application on an older system and run it on newer platforms. This could be seen as a base requirement for more free and Open Source Software: It's easier to acquire an older machine.

Unfortunately, transferring stuff from newer systems to older ones isn't that easy. The ''Savlib'' command writes stuff into a pre-created Save-File and records the release string of OS/400 where it was run. It is possible to set pre-set different values but the newer the utilized OS version, the newer the oldest possible compatibility level. V4R5 permits V3R2 as oldest release and select V4 releases. 
Transfers from old to new systems are painless, though.

== Single Level Storage ==
Since it's first incarnations the AS/400 creates a virtual 128 Bit address space for the upper OS layer and application programs. <ref>When looking a bit closer, the addresses are split into a 64 Bit actual ''address component'' and a 64 Bit numbered ''object type''. So, the claim of a 128 Bit address space aren't exactly wrong, but also not exactly right.</ref> This comprises physical memory and hard disk space.

A hard disk may be seen as a giant memory area mmap'ed into the address space of upper layer OS components. Swapping and Paging isn't done on the application layer as most conventional systems do, but on the address layer. Applications simply request contents from addresses and the layer below the OS called SLIC takes care of shuffling the data into RAM, should a page fault occur. Thus, page faults can occur also for data, not only for executables.

This concept also explains why the AS/400 platform lacks a conventional File System on it's disks. Since disks are essentially memory mapped into virtual memory space, everything visible is just an object at an address with certain properties. This allows to easily share these objects between requesting entities.

The advantage of this approach is the ability to do very fast context switches (if no or only a few page faults occur).<ref>According to Frank Soltis as written in his Books ''Inside the AS/400'', and ''Fortress Rochester''.</ref> This is the reason why even older machines with 100 % loaded CPU still have an astonishing fast response time when compared with common platforms of that era. If disk accesses are necessary for a context switch to take place, there's no notable speed difference to common platforms, though.

=== Disk Allocations ===
The usual way the system allocates space on secondary storage (aka: hard disks) is to spread objects over as many disks as possible. In modern RAID terminology, this is referred to as ''spindles'' (from the stack of rotating platters' axis), while IBM AS/400 documentation refers to the same concept as ''disk arms'': The actuator moving the read/write heads over the desired cylinder — a set of tracks — on the rotating platters.<ref>Once, disks with two actuators have been built, but these have been a development on the fringes and quickly vanished from the market.</ref>

This spreading of data has the same advantages and disadvantages as RAID 0:
* with increasing count of disk arms, the chance that a given data block is located on an idle drive increases and thus decreases latency through exploiting parallelism: the more applications run, the more data can be picked up from individual disks, the more overall processing work can be done.
* since data is located in the system just once, the failure of (at least) one disk has hard to predict ramifications. Objects become ''damaged'' because parts of them are no longer accessible.<ref>The necessity of a solid, and '''tested''' backup and restore concept is self-explanatory, but also emphasized in IBM documentation.</ref>

From early on, the concept of a dedicated I/O processor allowed for easy implementation of ''data mirroring'', to compensate for data loss because of a malfunctioning disk drive through redundancy.

Later, the well-known RAID-Levels have been made available through the availability of dedicated storage controllers.

=== Memory Pools ===
The described concept of the platform relying on paging as primary means to bring in data and code into the faster main memory for actual processing has some implications on main memory constrained machines. Once, RAM was extraordinarily expensive. Without precautions, it was easily possible to start many programs at the same time, each one competing for presence in RAM. When seriously overwhelmed with concurrent requests, this situation leads to [[wikipedia:Thrashing (computer science)|Thrashing]], a situation where the machine is completely busy with paging activity, but does no useful work anymore because of competing tasks.

OS/400 introduced the concept of memory pools for that purpose. A memory pool mainly has these attributes:
* Size,
* Shared or Private,
* Maximum Activity Level.

''Pool'' can be more or less translated to ''RAM Cache'': What's already in the pool's memory is ''cached'', because it mustn't be handled by the paging logic.

==== Pool Size ====
The ''Size'' of a memory pool is just how much RAM is set aside for applications being directed to run in this pool. More RAM means more cache and less disk activity for shoveling work into RAM.

==== Shared or Private Pool ====
''Shared pools'' allow more than one group of applications to utilize a memory pool, while ''private pools'' are reserved for just one group. See ''Subsystems'' below for the explanation of this grouping. This brings in another consideration: The more pools one creates, the more unused memory is left over from excess memory partitioning. <ref>OS/400 has a function called ''performance adjustment''. This function periodically checks paging activity and adjusts memory allocations between pools, as well as the pool's activity levels.</ref>

==== Maximum Activity Level ====
The ''Maximum Activity Level'' of a memory pool is the overall count of applications which are eligible to actively ''run'' in a given pool in parallel. This eligible count only considers applications which require actual access to the CPU, but not ''sleeping'' ones, waiting for some application-external event. This setting reduces competition for memory amongst applications within that pool. But at the same time — to a lesser extent — overall competition for main memory, system-wide.

==== Pool Summary ====
It is important to consider that this competition for RAM is not limited to actual code being in main memory, but also for data. Heavy paging can be quite normal for a batch job reading through a huge database file, and only a minor brake on other applications running. As long as the application code stays in main memory, that application can run again as soon as more data arrives in main memory to be processed. Because batch applications (ideally) use a different memory pool than interactive applications, there is no competition for main memory for these two application ''classes'' being confined in their respective pools.<ref>There ''is'' a certain degree of competition for disk I/O, though. As long as both pools need to access data on a common set of disks, I/O latency for both pools will go up, and response time for interactive applications will degrade. Again, this latency increase should happen most often for paging of data, not application code.</ref>

Many Books have been written about ''performance tuning'' the AS/400 system. As you can see, there are many aspects to consider which values of these settings lead to the sweet spot of a machine doing maximum (batch) work while minimally affecting interactive users. Dynamic scenarios have been described where interactive users are not considered at non-office hours, giving batch jobs more memory and thus allowing faster processing.

In current reality, there are two groups of users, and both of them rarely need to consider the points outlined above:
* Hobbyists are most often the sole users of a system. Thus, overall system activity levels are low — most often, the machine idles.
* Professional users with current systems have RAM in abundance. In addition, the increasing ubiquity of solid state storage with its intrinsic lack of mechanical positioning latency boosts I/O throughput.

For both groups, the directions given in the [[Post-Install Optimizations]] article gives an almost perfect starting point for the self-adjustment of the system itself to balance into a good partitioning of RAM into existing shared pools.

==== Expert Cache ====
In short, this is a setting of a pool where the system itself decides the block size of data being read from disk and put into memory. Without this, transferred blocks are of a fixed size, thus increasing processing overhead. <code>CHGSHRPOOL PAGING(*CALC)</code> switches Expert Cache on for a given pool. On the other hand, smaller data transfers decrease the wait time until some other disk access can be satisfied. 
Usually the recommended setting is ''enabled''. Allow the system to adjust itself. Note: The default setting is ''disabled''.

=== Subsystems ===
A subsystem is a way to group application programs. Subsystems are defined by a subsystem definition object. This object allows for many attributes to be set, to prepare the desired run-time environment. See [[Subsystems]] for a more in-depth illustration about the underlying concept.

=== Running Application Programs: Activation and Activation Groups ===
Since programs are already existent as objects in the single level store memory space, there's no need to load them from disk as common OS' need(ed) to do. To actually run such an application program needs some housekeeping, though. Commonly a program needs a [[wikipedia:Stack-based memory allocation|Stack]] and a [[wikipedia:Memory_management#Dynamic_memory_allocation|Heap]] for variables and other internal data structures to be able to run. The AS/400 is no different in this case, although this stuff is buried very deep into the LIC's innards.

Actually starting a program to be run on an AS/400 is called ''activation''. This task creates necessary storage as outlined above and adds the program to to the job scheduling queue. This is also necessary to provide a properly isolated environment when multiple users start the same application.

Completely different compared to common platforms, it's possible to ''share'' some of this space between programs. This shared space is called an ''activation group''. That is, while creating a new program (by compiling and linking), one can tell the compiler/linker if the program is to be run in
* the default activation group, that contains most of the OS,
* the caller's activation group, to allow a more dynamic approach, or
* a new activation group that is to be created for every instance the program has,
* a common activation group with a programmer specified name.

As long as any program of any activation group is activated, so is the activation group. Data structures will be destroyed only when the last program terminates, or ''the job ends'', <ref>A job is defined as work that needs to be done by the system. Work can be done by a single program or mutiple.</ref> or when a group is explicitly deallocated with the RCLACTGRP program. For details see weblinks.

Sharing an activation group shares (amongst other data):
* Static variables,
* Open files and file pointer positions,
* and more…

There's a certain intersection of functionality between a common activation group and <code>fork()</code>ing an application on Linux/Unix platforms. Even if it is not exactly the same but maybe it gives you an idea. 
On OS/400, programs to share an activation group must exist as separate objects while on Linux/Unix all code is either linked to a monolithic program or uses additional code with the aid of ''shared libraries''.<ref>A concept that ''also'' exists on the AS/400. There, they are called ''service programs''.</ref>

== Standardized UI ==
The whole UI revolves mainly around forms and menus, accompanied by static text. The layouts of these screens are somewhat standardized. AS/400 screens are very recognizable because of this.

See also: [[About Green Screens and mouse-clickable UIs]].

== Predictable Command Naming ==
For experienced users who won't repeatedly navigate though nested menus, commands may be entered on the command line instead of menu numbers. These commands consist of two or more parts, built of abbreviated english words. Abbreviations often take place by omitting vowels and shortening the result to less or equal four characters. The first one being a ''verb'' and the second one a ''subject'', optionally followed by more building blocks.

A few examples may illustrate the previous statements:
; wrksplf
: Work with Spooled Files
; chgmsgq
: Change Message Queue Properties
; crtdspf
: Create Display File
; dltf
: Delete File
; dspmsg
: Display Messages (from Message Queue)

See also [[wikipedia:IBM i Control Language|IBM i Control Language]] and [[CL Tricks]].

== Outstanding Reliability ==
Software is a complex thing built by humans. Humans make errors. Thus, Software contains errors. This is an inevitable fact.

IBM managed to make OS/400 not bug free but caught most problems that lead to a complete crash of the operating system. There may be edge cases that are usually hard to spot.

Also, IBM built multiple measures against ''defective sectors on hard disks being catastrophic'' into the OS from early on. Since there is no classical file system, most often, read errors only affect one or more objects that can be easily restored from backup.<ref>You don't have a backup? Own fault! No backup, no pity.</ref>

Reliability is '''not''' the same as availability. Some tasks can only be done when OS/400 is running in a restricted state.<ref>In Unix this would be called Single User Mode, with not much more than ''init'' and a root-shell running.</ref> This includes but is not limited to a full system backup or certain hardware failure recovery procedures.

== Rapid Development ==
OS/400 comes with a set of tools and supportive facilities within the OS to ease development of application programs. Usually, development comprises of:
* Definition and creation of database tables, display forms and printer output,
* Writing code that references these files and shovels data just by issuing proper READ and WRITE (among other) requests to these files.

The mentioned files are typed in a format called ''Data Definition Specifications'' or ''DDS''. It provides data field definitions, field definition references between files, arbitrary placement of static text (not for database files) and a multitude of flags for content to be shaped to fit for the desired presentation. 
The ''Screen Design Agent'' program (<code>strsda</code>) provides a pseudo-[[wikipedia:WYSIWYG|WYSIWYG]]-interface for creating screen forms and menus. 
The ''Report Layout Utility'' program (<code>strrlu</code>) provides a pseudo-WYSIWYG-interface for creating layouts for printer output files.

Actual programming can be done in RPG, COBOL, and C/C++. Output of the compilers (called ''modules'') created by any of these languages can be linked together to create an actual program object or a shared library (''service program''). Any language may call functions of any other language as long as parameters have been defined to be compatible.

The compilable CL "shell"-script language and REXX-Interpreter help in easing traditional automation tasks on the command line level.

Programming is done within the ''Program Development Manager'' collection of programs (<code>strpdm</code>). The most used component is the ''Work with File Members'' application, <code>wrkmbrpdm</code>, to list source code physical file content. This in turn calls the ''Source Entry Utility'' (''SEU'') with the appropriate parameters for actual code writing/editing with the desired existing or new file member.

''SEU'' provides a linetype-aware form for RPG and DDS, easing the matching of the appropriate columnns. It can be invoked as usual: With <code>F4</code>.

See also: [[How to program an Application]]

=== Times they are a changin' ===
Today, development tasks are most often done on Windows PCs, with IBM provided tools, that pull a local copy of a source file to edit.

Also, since the late 1990's, IBM has been trying to establish Java as ''the'' language of choice, along with helper programs for shifting away from the old fashioned character based applications (''green screen'') to a pure web-browser based way of user's interaction with the system.

Also, '''the''' AS/400 Language, RPG, has been enhanced and modernised slowly but constantly. While most examples in this collection of knowledge are based on ''ILE RPG IV'', newer OS releases were accompanied by compilers allowing to escape the punched card appeal of RPG more and more. V5 introduced the possibility to write actual code (not definition statements, though) in a free form, while with some V7 all parts of a RPG program may be written without being forced to write statements at a certain position within a line. This generates a load of syntactical changes to be learned.

Examples of positional code are slowly vanishing from Internet Websites. Looking up how to do stuff in positional code is getting increasingly difficult. Asking around in mailing lists for positional code examples sometimes yield unhelpful and cynic comments. 
Still, many companies have old to very old code to be maintained that predates compilers allowing free syntax. Providing mainly positional RPG examples is supposed to help understanding this old way of writing RPG, while examples for newer techniques are easily found with an internet search engine of choice.

== Weblinks ==
* [https://www.youtube.com/watch?v=5pY6Xxptp9A IBM AS/400 The Best Architecture In The World 1992], Video at YouTube.
* [https://www.mcpressonline.com/programming/rpg/introduction-to-ile-rpg-activation-groups Introduction to ILE RPG Activation Groups], McPress online.
* [https://www.mcpressonline.com/operating-systems/ibm-i-os400-i5os/i5os-v6r1-upgrade-now-or-later i5/OS V6R1: Upgrade Now or Later?], MC Press Online, with some background information regarding code observability.
* [https://www.itjungle.com/2020/11/23/frank-soltis-discusses-a-possible-future-for-single-level-storage/ Frank Soltis Discusses A Possible Future for Single-Level Storage], IT Jungle, also explaining the benefits of SLS for task switching.

== Footnotes ==
<references />

[[Category: Basic Knowledge]]
[[Category: Musings]]

Filesystems and Files on the AS/400

2026-04-14T14:15:00Z

Greenfive: fixed typo

{{FIXME|
* Add Chapter how to create physical and logical files,
* Move explanations to access paths to a programming article?}}
Since the AS/400 has been designed as a business helper machine, its processing capabilities mainly cover structured data with comparatively small record sizes. With the ongoing evolution of OS/400, more functionality was added by IBM that weakens this concise definition, though. On the other hand, the same added functionality improves ease of data storage for, and data exchange with other platforms.

== Different approaches of data storage and processing ==
Structured data can be processed most efficiently when the underlying logical and physical facilities support this approach natively instead of mimicking.

By comparison, most platforms outside the world of mainframes and minicomputers, as they were once known, operate on byte streams. Byte streams don't have any natural structure visible from outside. That is, without analyzing the content of the stream at any given moment, there is no chance to map the current byte position into any logical structure. 
This is somewhat similar to the differences between data transmission in an asynchronous manner (such as the RS-232 protocol specifies) versus synchronous. With asynchronous transfers, data payload has to be enriched with additional information to enable processing logic to chop the continuous stream of data into pieces to enable processing facilities to work on them in a structured way.

Most, if not all data is based on some structure, though. The simplest structure is a blob of data of a given size, usually named a ''block''. When looking closely, conversion from, and to byte streams from and to blocks takes place on many levels:
* Storage of data on hard disks or in solid state memory in blocks of 512 Bytes or 4 [[wikipedia:Kibibyte|KiB]],
* Paging and Swapping activity of virtual memory systems also takes place in blocks instead of streams,
* Data Transfer over todays networks is done in pieces called ''frames''.<ref>To be precise, this isn't done to ease or enable record oriented processing but to enable sharing of physical media (like cables) between many participants of the network.</ref>

On a higher level of a file's content definition, more elaborated structuring of stream-based data can be found:
* Pixel-oriented picture files include a header that dynamically defines the format of the data that follows,
* Movie files are defined by different kinds of data blocks: To save space, many frames are only saved as differences to the picture before,<ref>From time to time, a key frame contains a complete picture to compensate for data loss, that can happen for many reasons.</ref>
* Todays most used text processing software internally uses [[wikipedia:XML|XML]] to give seemingly unstructured data a structure,
* Application Program Files also are structured in a way that permits the operating system to efficiently load and execute the contained code,
* Even plain text files have a structure: They contain of words, sentences, and paragraphs.

The mapping of structured data into byte streams leverages processing speed with flexibility to store arbitrary complex data structures into very simple data structures on disk. Today, the processing overhead of mapping byte streams into records (blocks) or more elaborated data structures is almost nullified by cheap availability of sheer processing power.

=== Storage Overhead ===
However, mapping byte streams into logical blocks of information often creates ''storage overhead''. For example, a plain text file with the word 'Hello' uses only 5 Bytes of actual data<ref>Counted as ASCII, or SBCS as named in the AS/400-World.</ref> (sometimes followed by implicit end of line marker(s)), the file occupies at least 512 Bytes data, since this is the minimum block size on usual hard disks. If the file is expanded to exactly 512 Bytes by adding more text, the overhead is zero. Adding just one character starts the continuation of data in the next disk block and again occupies the whole block of that 511 Bytes are wasted. 
This overhead might be negligible when looking at todays file sizes. But, every OS utilizes configuration files, that usually are small and fit the above description perfectly. ''/etc/hosts'' may be 680 Bytes long on a given system, that makes up 344 Bytes wasted space, according to the above calculation.<ref>Actually, with todays partition sizes, Linux uses 4 KiB blocks, not 512 Bytes. This raises the overhead of said ''/etc/hosts'' to 3416 Bytes.</ref>

Today, this storage overhead is also almost nullified by cheap availability of gigantic hard disk storage capacity.

=== Latency ===
Considering the above description, it's easily observable that actual data is scattered across a greater area of a given hard disk with many holes in between, compared to all files concatenated to one big blob. This increases latency, because the disk's head assembly must move more often to access a certain block on the physical media.

Today, I/O latencies are nullified by cheap availability of solid state memory “disks”.

== AS/400 IFS ==
Early incarnations of the AS/400 had no facilities to save or process byte-based ''stream files''. There only was the possibility to create files with a fixed record structure. With the introduction of the ''integrated file system'' in OS/400 Version 3, saving and processing of byte stream files has been made possible. The classical record oriented file system was in turn named ''QSYS.LIB''-file system and looks like a classical Unix mount point, when looking with applications being aware of the IFS, such as ''qsh'', the minimal UNIX like shell. The rest of the filesystems allow to store stream files. 
Up to today, ''QSYS.LIB'' is still the most important file system on AS/400's because most part of the OS is stored there, as well as configuration database files and, of course application specific database files.

This ''IFS'' is not the same as file systems in Linux or other common operating systems. There, one or more disks are ''partitioned'': Space from one disk split into slices of usable space by software. To make use of the space available in this partitions, a ''file system'' has to be created. That is, database-like structures for the mapping of files to disk blocks are written into any given partition space. The space available for the creation of files on any given file system is thus fixed to the size of the partition, minus overhead, minus space needed for the aforementioned data structures. Multiple filesystems have been conceived over decades with a determined set of features, so it's possible to use different file systems to match the different storage needs of different types of files.

On the other side, the IFS of the AS/400 is a purely ''logical structure'' that allocates disk blocks as requested by the operating system. If a record oriented file in the classical ''QSYS.LIB''-filesystem is filled with data, disk blocks are filled with this data and are thus no longer available for storage. If someone transfers a picture file to the AS/400's ''/QOpenSys''-Filesystem, disk-blocks are allocated for that data and also cannot be allocated for storage. There is no static assignment for these different file systems to partitions or disk blocks.

'''Stuff saved in file systems on the AS/400 is usually referred to as ''objects''.

== AS/400 files ==
The AS/400 file handling shares a common heritage of file handling found in IBM mainframes (MVS or z/OS partitioned data sets). An AS/400 ''file'' defines at least the record length. All ''members'' of these file share this maximum length and thus are constrained to this length. 
In addition, an ''externally defined file'' can be more strict than an MVS PDS, by not only enforcing a record length, but also an actual field layout and -content over this record. <ref>The term ''external'' is greatly misleading here, because the file definition is ''external'' to the ''program'' using that file. Indeed, the field layout is ''internal'' to the file object.</ref>
This can be roughly compared to a directory on Linux or (better) a tar file, where one may only add TIFF image data with a common horizontal pixel count.<ref>This is fairly unpractical in computing as we know it.</ref>

For example, by creating a ''source file'' on an AS/400, one defines a ''file'' with a certain record length.<ref>92 Bytes by default.</ref> One may use this ''file'', create ''members'' and add program source code to these members at will, but it's not possible to add data with any line longer than 92 characters into any member of that file.

=== File Types ===
Basically there are four types of files found in any AS/400's classical ''QSYS.LIB'' filesystem:
* '''Source Files''' are ''physical files'', aka ''database files'' with at least two fields per record:
** Line of Text, primarily to hold program code to be compiled,
** Timestamp, records the last change of that particular line.<ref>SEU shows these timestamps on the very far right of it's display.</ref>
* '''Physical Files''' are again database files but with a user defined record length and field format.
* '''Logical Files''' are files that reference one or more physical files, but do not contain any data. LFs can be compared to the SQL terms:
** ''view'': Show only a subset of fields of PF,
** ''join'': Show a superset of fields of more than one PF,
** ''index'': Apply different indexes, and thus record sorting.
: In addition, restrictions about which records to select or omit from the PF can be added.
* '''Other Files''' that appear as type ''*FILE'' in output listings that include, but are not limited to
** ''Display Files'' that map data to positions on the screen and enable user interaction to this forms,
** ''Printer Files'' that map data to positions on a virtual sheet of paper for later printer output,
** ''Inter Communication Facility Files'' who provide an easy way for application programs to talk to other programs on the same or remote AS/400's,
** ''Save files'' that provide roughly the same functionality as tar files on Linux, enabling bundling of files (objects) of different types into one self-contained archive, for easier transmission to other systems for re-extracting or permanent storage.

Handling of these files will be explained in separate articles.

==== A word on physical and logical (database) files ====
The main usage of logical files is to provide different ''access paths'' compared to the linked physical file to read data. An access path is just one or more [[wikipedia:Database index|database index(es)]] created for one or more fields of a given physical file.

When accessing database files through the classical READ(P)(E)-API-Calls, there's no possibility to sort a result set dynamically as it's easily done with [[wikipedia:SQL|SQL]]. There's not even a result set. There's just kind of a ''file pointer'' that points to the last read row, and the order in that the rows are retrieved by subsequent READs (that increment this file pointer) are determined by the access path(s). 
Keep in mind that this handling of structured data stems from a time period when computers had only a tiny fraction of today's processing power<ref>Processing as in ''Read from disk — Modify — Write to disk''.</ref> and storage capacity.

Considering the clear positioning of the AS/400 as ''database machine'' by IBM, and adding the fact that the precursor to SQL was created in the early 1970s by IBM employees, SQL might have been available on the AS/400 from it's first release. 
On older machines, SQL proves to be painfully slow, especially when dealing with huge database tables. Since the SQL engine dynamically adds database indexes as needed when processing queries, getting data out of a table can be really slow when such a temporary index must be created in advance.

On the other side, utilizing the legacy API-Calls to retrieve and process random records with CHAIN-READ from multi-GiB-sized database tables is surprisingly quick on old machines for many reasons.

This somewhat restricted API call interface explains why data handling within applications on the AS/400 feel so fundamentally different compared to what we're used to.

Most common platforms utilize SQL:
* The user is providing (part of) a query string, the database picks matching rows and
* the application delivers a limited view of the whole database content back to the user.
** The user may easily change sorting of the result records to be based on any single field by usually clicking on the respective table header field in the application's UI,
* the user may finally select the desired record for further processing.

AS/400 applications usually…
* provide a complete list view<ref>These lists are called ''subfiles''.</ref> of the entire database content, with a fixed sort order. The user may either scroll to the desired record, or type part of a search string in a designated input line and press <code>Enter</code>.
* The application reloads the list view after CHAINing (placing the ''file pointer'') to the first record matching (part of) the search string<ref>To be precise, the search string is matched against the '''start''' of the respective strings in the key field. It's not a <code>… LIKE '%FOOBAR%'</code>-matching that is simply not possible with the legacy API.</ref>. The rest of the list is filled with subsequent READs until the list is full or [[wikipedia:End-of-file|EOF]] is reached.
** Skillful programming techniques allow the user to freely scroll forward and backward from the record retrieved in the previous step.
* The user may finally select the desired record for further processing.

Programs don't need to maintain the list programmatically xor allocate memory for loading the whole database. Details will be explained in the articles about programming.

== Footnotes ==
<references />

[[Category: Basic Knowledge]]

Using NFS-based image catalogs for backups

2026-04-05T20:42:33Z

Greenfive: fixed typo

'''Using NFS-based image catalogs for backups''' is an alternative and inexpensive way to use OS provided facilities for backups. If a single tape drive would suffice for a given backup scenario, but is not viable because the machine is located in a remote location, NFS-based image catalogs might be worth considering.

<blockquote>'''Note:''' The procedures outlined below have been tested on IBM i 7.2 and 7.3. V4R5 definitely has no support for image catalogs.</blockquote>

Pro:
* No temporary local data,
** No local I/Os due to data to be backed up being copied into the backup images,
* No unnecessary transfers of image files bloated to the maximum size to the final file destination.

Con:
* Full restores require a D-mode manual IPL from the last ''Save 21''. This can be handled through copying that image to the HMC or VIOS storage, so it can be tied to an emulated optical drive. Depending on the image size, this simply might not fit onto the available space, or takes unacceptably long to be copied. It should be possible to import the NFS exported backup directory to VIOS as an installation image collection to spare the lengthy copy, but this has not yet been tested.

== Configuring the service tools adapter on IBM i ==
On IBM i, you need to establish a ''service tools adapter''. This is a virtual network IOA which is just added to the LPAR and otherwise not referenced in the OS with a line description. Configuration of this adapter is done in DST.<ref>Configuration in SST is also possible, but the menu item numbers and labels are slightly different.</ref>:
* <tt>5. Service tools device IDs</tt>,
* <tt>F13</tt>,
* Select one of the adapters shown with <tt>1</tt>.<ref>We recommend the one with the numerically higher ''Resource Name''. This is a purely cosmetic decision.</ref>
* Enter the IP configuration,
* <tt>F7</tt>,
* <tt>F17</tt>; Ping to this address from another machine connected to the network should now work.

NFS traffic for image catalogs exclusively runs through this adapter, being managed by SLIC. The "upper layer" OS components are not involved. Access to NFS catalogs is working in restricted mode, with TCP/IP being unavailable.

== Configuring the NFS-Server ==
This setup guide assumes a Linux based NFS server as backup destination.
* Install NFS server,
* configure NFS server by editing ''/etc/exports'',
* reload configuration by running <code>exportfs -ra</code>.

Example entry:
/backups/i-backups 192.168.0.10/32(rw,no_root_squash,no_subtree_check,async)

192.168.0.10 is the address you gave to the service tools adapter in the previous step. This assures that only that IP address can actually access backup images.

The ''image catalog'' is in fact a simple ASCII text file. It
* must be named ''VOLUME_LIST'',
* has one entry per optical image in the same directory,
* optionally has a blank and the letter ''W'' to mark the respective image as writeable,
* cannot be managed through the usual image catalog commands on IBM i.

Example entry:
20250528-1.udf W

Using sparse files as images works and is highly recommended. It saves time and unneeded I/O to create the images compared to creating preallocated images filled to the brim with zero bytes. Such files can be created with
truncate -s 4G /backups/i-backups/20250528-1.udf

This quickly creates a file with initially 0 Bytes in size, with a maximum size of 4 GiB. The suffix ''-1'' designates this file as a differential backup to the last full save.

You can use this example shell script to create images with ''cron'' on Linux before the backup job runs on IBM i:
<syntaxhighlight lang=shell>
#!/bin/bash

TODAY="$(date '+%Y%m%d')"
SIZE="4G"
DIR="/backups/i-backups"

if [ ! -f "${DIR}"/"${TODAY}"-1.udf ]; then
truncate -s "${SIZE}" "${DIR}"/"${TODAY}"-1.udf

if [ ! -f "${DIR}"/VOLUME_LIST ]; then
echo "${TODAY}-1.udf W" > "${DIR}"/VOLUME_LIST
else
sed -i "1i ${TODAY}-1.udf W" "${DIR}"/VOLUME_LIST
fi
fi

exit 0
</syntaxhighlight>
You can name it ''create-imgclge-daily''.

The image catalog logic on IBM i automatically loads the first entry in ''VOLUME_LIST''. Hence the shown script inserts new images at the beginning of the file.

== Configuring IBM i ==
With remote, NFS-based image catalogs, an image catalog is tied to a device file. We create the device ''optbkup01'' for that purpose:
crtdevopt devd(optbkup01) online(*no) rsrcname(*vrt) lclintneta(*srvlan) rmtintneta('192.168.0.11') netimgdir('/backups/i-backups')

192.168.0.11 is the IP address of the NFS server.

Next, we can vary on the device:
vrycfg cfgobj(optbkup01) cfgtype(*dev) status(*on)

Finally, we can look at the catalog entries:
wrkimgclge imgclg(*dev) dev(optbkup01)

Always make sure that you vary off the device after usage, so changes to ''VOLUME_LIST'' will be recognized at next vary on.

== Differential saves ==
Running an incremental save basically runs these steps:

* Create new image, and update ''VOLUME_LIST'' on the NFS server
* Vary on the backup device, and initialize the automatically attached first image
* ''savchgobj'' — Back up changed objects from user libraries since the last full backup
* ''savcfg'' — Back up configuration objects
* ''savsecdta'' — Back up user and access list objects
* ''savdlo'' — Back up Document Library Objects (DOS 8-character compatibility file system)
* ''sav'' — Back up IFS
* vary off the backup device

The following CL program automates these steps:
<syntaxhighlight>
PGM /* Save Changed Stuff to an Image Catalog. */
DCL VAR(&OPTDRV) TYPE(*CHAR) LEN(10) VALUE(OPTBKUP01)
DCL VAR(&TODAY) TYPE(*CHAR) LEN(6)
DCL VAR(&OPTDAT) TYPE(*CHAR) LEN(10)
/* Vary on and initialize with current date. */
RTVSYSVAL SYSVAL(QDATE) RTNVAR(&TODAY)
CVTDAT DATE(&TODAY) TOVAR(&OPTDAT) TOFMT(*YMD) +
TOSEP(*NONE)
VRYCFG CFGOBJ(&OPTDRV) CFGTYPE(*DEV) STATUS(*ON)
DLYJOB DLY(15)
INZOPT DEV(&OPTDRV) NEWVOL(&OPTDAT) MEDFMT(*UDF) +
VOL(*MOUNTED) CHECK(*NO) THRESHOLD(*CALC)
/* Actually save. */
SAVCHGOBJ OBJ(*ALL) LIB(*ALLUSR) DEV(&OPTDRV) +
OBJJRN(*YES) VOL(&OPTDAT) ENDOPT(*LEAVE) +
UPDHST(*NO) PRECHK(*NO) SAVACT(*SYNCLIB) +
SAVACTWAIT(30) DTACPR(*LOW) +
OMITLIB(SAVRST) OUTPUT(*PRINT)
MONMSG MSGID(CPF3774 CPF3778 CPF9845)
MONMSG MSGID(CPF3794) EXEC(GOTO CMDLBL(SENDMSG))
SAVCFG DEV(&OPTDRV) ENDOPT(*LEAVE) DTACPR(*LOW) +
OUTPUT(*PRINT)
SAVSECDTA DEV(&OPTDRV) ENDOPT(*LEAVE) DTACPR(*LOW) +
OUTPUT(*PRINT)
SAVDLO DLO(*CHG) DEV(&OPTDRV) ENDOPT(*LEAVE) +
OUTPUT(*PRINT) DTACPR(*LOW)
MONMSG MSGID(CPF906B)
MONMSG MSGID(CPF6772) EXEC(GOTO CMDLBL(SENDMSG))
SAV DEV(('/QSYS.LIB/' *CAT &OPTDRV *TCAT +
'.DEVD')) OBJ(('/*') ('/QSYS.LIB' *OMIT) +
('/QDLS' *OMIT) ('/QNTC' *OMIT) +
('/QFileSvr.400' *OMIT)) OUTPUT(*PRINT) +
CHGPERIOD(*LASTSAVE) UPDHST(*NO) DTACPR(*LOW)
MONMSG MSGID(CPF3837 CPD37C3 CPFA09E CPD384E)
MONMSG MSGID(CPF6772) EXEC(GOTO CMDLBL(SENDMSG))
GOTO CMDLBL(END)
SENDMSG:
SNDMSG MSG('Fehler aufgetreten: Sicherungsabbruch.') +
TOUSR(QSYSOPR)
END:
DLYJOB DLY(15)
/* Close device file properly */
VRYCFG CFGOBJ(&OPTDRV) CFGTYPE(*DEV) STATUS(*OFF)
ENDPGM
</syntaxhighlight>

This CL code can be saved into e. g. ''qgpl/qclsrc.sav2imgclg'', compiled, and run through ''wrkjobscde'' after the cronjob above has created the new images.

== Full saves ==
These are also called ''Save 21'', because it is run through item 21 in the ''save'' menu. The procedure is very much akin to incremental saves, but there are no individual calls to the various save programs.

<syntaxhighlight>
PGM /* Vary on and inzopt optbkup01. */
DCL VAR(&OPTDRV) TYPE(*CHAR) LEN(10) VALUE(OPTBKUP01)
DCL VAR(&TODAY) TYPE(*CHAR) LEN(6)
DCL VAR(&OPTDAT) TYPE(*CHAR) LEN(10)
/* Vary on and initialize with current date. */
RTVSYSVAL SYSVAL(QDATE) RTNVAR(&TODAY)
CVTDAT DATE(&TODAY) TOVAR(&OPTDAT) TOFMT(*YMD) +
TOSEP(*NONE)
VRYCFG CFGOBJ(&OPTDRV) CFGTYPE(*DEV) STATUS(*ON)
DLYJOB DLY(15)
INZOPT DEV(&OPTDRV) NEWVOL(&OPTDAT) MEDFMT(*UDF) +
VOL(*MOUNTED) CHECK(*NO) THRESHOLD(*CALC)
MONMSG MSGID(OPT1330) EXEC(GOTO CMDLBL(SENDMSG))
GOTO CMDLBL(END)
SENDMSG:
SNDMSG MSG('Fehler aufgetreten: Abbruch.') +
TOUSR(QSYSOPR)
END:
ENDPGM
</syntaxhighlight>

This CL code can be saved into e. g. ''qgpl/qclsrc.prepsav21'', compiled, and run manually after the you manually have created the new full save image.

Since Save 21 is an inherently manual process, four steps are necessary:
* Create the new full save image, and an appropriate entry in ''VOLUME_LIST'':
<syntaxhighlight lang=shell>
truncate -s 20G /backups/i-backups/$(date '+%Y%m%d')-0.udf
sed -i "1i $(date '+%Y%m%d')-0.udf W" /backups/i-backups/VOLUME_LIST
</syntaxhighlight>
* Vary on, initialize the image by running ''prepsav21'' shown above.
* <tt>go save</tt>, type 21 and press return.
* After completion, vary off the device.
vrycfg cfgobj(optbkup01) cfgtype(*dev) status(*off)

Always make sure that you vary off the device after usage, so changes to ''VOLUME_LIST'' will be recognized at next vary on.

== Single-file restore ==
* Vary on image catalog:
vrycfg cfgobj(optbkup01) cfgtype(*dev) status(*on)

* If necessary, select backup image (by date), and load:
wrkimgclge imgclg(*dev) dev(optinst01)

* For each backed up library, a file with the same name as the library was created in the respective image. To see what's in it:
dspopt vol(250417) data(*savrst) path('/mylib')

* Since ''savchgobj'' is used for backup, ''rstobj'' must be used for restoration, e. g.:
rstobj obj(*all) savlib(mylib) dev(optbkup01) endopt(*leave) mbropt(*all) rstlib(savrst) output(*print) optfile('/mylib')

* Vary off the image catalog:<ref>If this is not done, the subsequent automatic incremental backup will fail.</ref>
vrycfg cfgobj(optbkup01) cfgtype(*dev) status(*off)

== Pitfalls ==
* ''VOLUME_LIST'' is referenced only when varying on the optical device on IBM i. Changes to the file while the device is varied on are not recognized.
* Each file referenced in ''VOLUME_LIST'' '''must''' exist. If not the whole image catalog will behave erratic. ''Inzopt'' will fail with error code <code>OPT1300</code>. On the other hand, the directory ''may'' contain files not being referenced in the image catalog. These are invisible to the image catalog logic.
* <tt>ls -l</tt> on the NFS server always shows the maximum allocatable size of the images. If you want to know the actually used size, use <tt>du</tt>.

== Weblinks ==
* [https://www.ibm.com/docs/en/i/7.3?topic=recovery-recovering-your-system Recovering your system], IBM.com
* [https://www.ibm.com/support/pages/ibm-i-save-and-restore-using-virtual-optical-images-nfs-server IBM i save and restore using virtual optical images on an NFS server], IBM.com
* [https://archive.midrange.com/midrange-l/202012/msg00377.html Image Catalog over NFS], discussion on midrange.com

== Footnotes ==
<references />

[[Category: System Administration‏‎]]

Keeping your machine clean

2026-04-05T15:45:21Z

Greenfive: fixed typo

OS/400 has a lot of facilities to keep your machine running smoothly without doing anything sysadmin related. I want to not only keep the machine running smoothly, but keep it clean und uncluttered by unnecessary data. Thus I sign on as ''QSECOFR'' on a daily basis and check/clean the following:

* Output queues (<code>WRKOUTQ</code>), purging job logs and other output no longer relevant.
* Messages (<code>DSPMSG</code>) for the users ''QSYSOPR'', ''QTCP'', ''QPGMR''. These get purged by the system cleanup jobs eventually, but retain some messages. To keep overview about more or less important things happening, I want at least have a look at new messages appearing in ''QSYSOPR''s message queue.
* Alerts (<code>WRKALR</code>) are a more persistent, and visible way for the system to record possibly important messages, which might be easily overseen by just (loosely over)looking routinely over the ''QSYSOPR'' messages.
* Problems (<code>WRKPRB</code>) are the next step to alerts. Some things on the system trigger all three: Messages, Alerts, and Problem entries.

[[Category: System Administration]]

Tape Drives on old AS/400

2026-04-05T15:25:33Z

Greenfive: fixed another typo

The QIC drives contained in many older machines are often defective, or it's expensive to impossible to get actual tapes for these drives. Sometimes, stock tape drives can be used also. This article documents which combination of machine, OS release, and drive has been tested so far.

<blockquote>'''Note''': The mentioned drives have been used to '''replace''' an existing internal drive, being connected to the internal SCSI bus. Some machines have hidden SCSI ports. Using those will fail to terminate the SCSI bus properly, with accompanying problems for the connected devices to transfer data.</blockquote>

{|class="wikitable sortable"
!Drive
!Server/OS
!Tasks tested
!Remark
|- style="vertical-align:top; background-color:#ddffdd;"
|HP DDS-1/2/3
|9401-150/V4R5
|Save/Restore/BMR<ref>''B''are ''M''etal ''R''estore, IPL from tape and restore of the entire system.</ref>
|DIP-Switches set to default.
|- style="vertical-align:top; background-color:#ddffdd;"
|HP DLTVS 80<ref>'''Caution!''' Protect the controller board of the drive with a piece of carton or other insulating material. The EMI prevention springs of the case will scratch components from the board otherwise.</ref>
|9406-S20/V4R4 + 2726 SCSI RAID-Controller
|Save/Restore
|SCSI-Error logged at IPL time. Works otherwise.
|- style="vertical-align:top; background-color:#ddffdd;"
|Dell LTO2<ref>Reported by third party.</ref>
|9406-170 V4
|Save/Restore
| 
|- style="vertical-align:top; background-color:#ffffdd;"
|Conner CDT20004R-S
|9401-150/V4R5
|Save/Restore
|DIP-Switches set to default. SCSI errors are logged for every initial access. Works otherwise.
|- style="vertical-align:top; background-color:#ffffdd;"
|HP LTO1
|9406-800/V5R4 + 2757 SCSI RAID-Controller
|Save/Restore
|SCSI-Errors logged at IPL time, and for every initial access. Works otherwise.
|- style="vertical-align:top; background-color:#ffffdd;"
|Quantum DLT4000
|9401-150/V4R5
|Save/Restore
|SCSI-Error 63A09210<ref>Illegal or unsupported tape unit response.</ref> logged at IPL time, and for each access. Works otherwise.
|- style="vertical-align:top; background-color:#ffdddd;"
|HP DDS-1/2/3
|9402-400/V4R2
|None.
|DIP-Switches set to default. Drive not recognised in DST.
|- style="vertical-align:top; background-color:#ffdddd;"
|HP DDS-1/2/3
|9401-P03/V3R2
|None.
|DIP-Switches set to default. Drive not recognised in DST.
|}

Color legend:
{|class="wikitable
|style="align:center; background-color:#ddffdd;"|Green is for ''works without or with just minor errors''.
|style="align:center; background-color:#ffffdd;"|Yellow is for ''works, but with errors''.
|style="align:center; background-color:#ffdddd;"|Red is for ''does not work''.
|}

Please see the ''Remarks'' column for further details.

== A Word on HP DDS drives ==
[[File:Dds-roller.jpg|thumb|right|Paper-Sheet type cleaning roller (red arrow)]]
All those drives feature an automatic head cleaning mechanism.

Certain (newer?) models have kind of a roller made from tiny raspy paper sheets, which is momentarily pressed to the rotating head drum via a solenoid. Other (older?) drive models feature a foam roller instead of these paper sheet cleaners. The foam has over the years degraded into bitumen foam. Once it made contact with the head from a test run, you'll have with that sticky substance forcing the tape being pulled out of the cartridge and winding around the drum in very high speed. A lot of work to undo.

For these drives, '''it's crucial to prevent the foam roller to make contact with the head before you do any testing'''!
* by physically removing the foam residue from the roller; denatured alcohol helps in cleaning.
* by unplugging the power plug to the coil.
* by removing that cleaning assembly completely.

I have two drives running without being able to clean automatically for years. They tend to more often ask for a cleaning tape, though. And once in a while, I remove the drives and clean the drum assembly by hand, with the tip of a soft cloth soaked with denatured alcohol.

I have not yet found a reliable way of determining which DDS1 and 2 drive models are "safe" and which are not. I have not yet had a DDS3 drive out of four with that foam roller.

== See also ==
* [[Tape device related system reference codes]]

== Footnotes ==
<references />

[[Category: Hardware Configuration Guide‏‎]]

Tape Drives on old AS/400

2026-04-05T15:25:03Z

Greenfive: fixed typo

The QIC drives contained in many older machines are often defective, or it's expensive to impossible to get actual tapes for these drives. Sometimes, stock tape drives can be used also. This article documents which combination of machine, OS release, and drive has been tested so far.

<blockquote>'''Note''': The mentioned drives have been used to '''replace''' an existing internal drive, being connected to the internal SCSI bus. Some machines have hidden SCSI ports. Using those will fail to terminate the SCSI bus properly, with accompanying problems for the connected devices to transfer data.</blockquote>

{|class="wikitable sortable"
!Drive
!Server/OS
!Tasks tested
!Remark
|- style="vertical-align:top; background-color:#ddffdd;"
|HP DDS-1/2/3
|9401-150/V4R5
|Save/Restore/BMR<ref>''B''are ''M''etal ''R''estore, IPL from tape and restore of the entire system.</ref>
|DIP-Switches set to default.
|- style="vertical-align:top; background-color:#ddffdd;"
|HP DLTVS 80<ref>'''Caution!''' Protect the controller board of the drive with a piece of carton or other insulating material. The EMI prevention springs of the case will scratch components from the board otherwise.</ref>
|9406-S20/V4R4 + 2726 SCSI RAID-Controller
|Save/Restore
|SCSI-Error logged at IPL time. Works otherwise.
|- style="vertical-align:top; background-color:#ddffdd;"
|Dell LTO2<ref>Reported by third party.</ref>
|9406-170 V4
|Save/Restore
| 
|- style="vertical-align:top; background-color:#ffffdd;"
|Conner CDT20004R-S
|9401-150/V4R5
|Save/Restore
|DIP-Switches set to default. SCSI errors are logged for every initial access. Works otherwise.
|- style="vertical-align:top; background-color:#ffffdd;"
|HP LTO1
|9406-800/V5R4 + 2757 SCSI RAID-Controller
|Save/Restore
|SCSI-Errors logged at IPL time, and for every initial access. Works otherwise.
|- style="vertical-align:top; background-color:#ffffdd;"
|Quantum DLT4000
|9401-150/V4R5
|Save/Restore
|SCSI-Error 63A09210<ref>Illegal or unsupported tape unit response.</ref> logged at IPL time, and for each access. Works otherwise.
|- style="vertical-align:top; background-color:#ffdddd;"
|HP DDS-1/2/3
|9402-400/V4R2
|None.
|DIP-Switches set to default. Drive not recognised in DST.
|- style="vertical-align:top; background-color:#ffdddd;"
|HP DDS-1/2/3
|9401-P03/V3R2
|None.
|DIP-Switches set to default. Drive not recognised in DST.
|}

Color legend:
{|class="wikitable
|style="align:center; background-color:#ddffdd;"|Green is for ''works without or with just minor errors''.
|style="align:center; background-color:#ffffdd;"|Yellow is for ''works, but with errors''.
|style="align:center; background-color:#ffdddd;"|Red is for ''does not work''.
|}

Please see the ''Remarks'' column for further details.

== A Word on HP DDS drives ==
[[File:Dds-roller.jpg|thumb|right|Paper-Sheet type cleaning roller (red arrow)]]
All those drives feature an automatic head cleaning mechanism.

Certain (newer?) models have kind of a roller made from tiny raspy paper sheets, which is momentarily pressed to the rotating head drum via a solenoid. Other (older?) drive models feature a foam roller instead of these paper sheet cleaners. The foam has over the years degraded into bitumen foam. Once it made contact with the head from a test run, you'll have with that sticky substance forcing the tape being pulled out of the cartridge and winding around the drum in very high speed. A lot of work to undo.

For these drives, '''it's crucial to prevent the foam roller to make contact with the head before you do any testing'''!
* by physically removing the foam residue from the roller; denatured alcohol helps in cleaning.
* by unplugging the power plug to the coil.
* by removing that cleaning assembly completely.

I have two drives running without being able to clean automatically for years. They tend to more often ask for a cleaning tape, though. And once in a while, I remove the drives and clean the drum assembly by hand, with the tip of a soft cloth soaked with denatured Alcohol.

I have not yet found a reliable way of determining which DDS1 and 2 drive models are "safe" and which are not. I have not yet had a DDS3 drive out of four with that foam roller.

== See also ==
* [[Tape device related system reference codes]]

== Footnotes ==
<references />

[[Category: Hardware Configuration Guide‏‎]]

How to program an Application

2026-04-04T10:38:53Z

Greenfive: fixed typo

{{FIXME|Finish article.}}
Getting started on programming the AS/400 isn't hard. According to Frank Soltis, the most effective way to get a machine sold is to actually bring an AS/400 along to the future customer, and show an C''x''O in a one hour crash course how to code a simple ''business application''. 
Well, something an arbitrary business buffoon can do, you can do, too. Since the usual [[wikipedia:"Hello, World!" program|Hello World]] is just too lame, let's start with an actually usable example: A simple address book application.

== Components of a typical Program ==
Since we're talking about a database machine, the first we need to know is how to create ''Database Tables''.

Editing Database Content is usually done in screen ''forms''. These are called ''Display Files''. So, let's also learn how to create Display Files in the next turn.

Finally, there must be a program for handling key presses from the user and coping with the I/O of database- and display files.

Optionally, there can be more files to support additional functions in a program. Printer files allow for formatted output (tabular report) to a printer. InterSystem Communication Files allow for Record-I/O to other programs on the local or a remote host. We will not dig into these, for now.

=== Programming Languages ===
Available on a full install of an older OS/400 are compilers for
* [[wikipedia:C (programming language)|C]]/[[wikipedia:C++|C++]],
* [[wikipedia:COBOL|COBOL]],
* [[wikipedia:IBM i Control Language|CL]], the equivalent of batch/shell scripts,
* [[wikipedia:Java (programming language)|Java]],
* [[wikipedia:IBM RPG|RPG]],<ref>Report Program Generator.</ref>

and interpreters for
* [[wikipedia:Rexx|REXX]], a scripting language derived from the mainframe world.

RPG and COBOL are available in two flavors: [[wikipedia:Integrated Language Environment|ILE]] and [https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_73/ilec/opmdesc.htm OPM]. See linked explanations for details.

It makes no sense to utilize OPM compilers for new projects. It's best you save space and deinstall these as well as the ''*PRV'' variants, that generate code for the immediately previous OS release. We'll solely focus on ILE since it has (amongst others) the advantage of linking (binding) compiled object code from different languages together to form a program.

The IBM website features [https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzahg/rzahglanguages.htm extensive documentation] for each language.

== A Word on Editors ==
If you are familiar with ''vi'' or ''vim'' on Unix-like platforms, learning SEU will be an easy task. You may alternatively utilize an editor of your choice on the platform of your choice.<ref>Windows Notepad is '''not''' a proper editor. It's sufficient for notes, though. Hence the name.</ref> To help you chose, we'll take a closer look on the pros and cons of each solution.

{|class="wikitable"
!Editor
!Pro
!Con
|-
|valign="top"|'''SEU on OS/400'''
|valign="top"|Built-in contextual support for certain file types and kind of lines with precise column requirements. Editing is done in an implicit array of blanks in permanent overwrite mode for easier placement of characters in proper position.
|valign="top"|Can display only a few lines of code, that reduces clarity and thus forces a lot of scrolling. Uniform colouring of code is increasingly uncommon and makes it even more difficult to keep an overview.
|-
|valign="top"|'''Other on other platforms'''
|valign="top"|Familiar operation and UI. Easy content duplication with clearly visible text range selection, and copy-paste. Possibility to create custom rules for syntax highlighting.
|valign="top"|Cumbersome handling: After every change that should result in a compile run, changed files have to be transferred back in some way.
|}

The Killer-Feature of SEU is the extensive contextual support it provides for RPG and DDS, and partly for COBOL. For beginners, it's much easier to utilize this helpful function often until some knowledge, routine and working programs as copy-source have been established.

=== Vim ===
Vim is most often the editor of choice for seasoned Text-UI aficionados. Since ''vim'' is able to open files on remote-locations via URL notation, you can perfectly use it for editing on the fly from any platform vim is available for. There are even extensions for [https://github.com/kaimoku/vim-syntax-for-i syntax-highlighting] of common code.<ref>No longer working with vim 8.</ref> Automation of login can be done via [http://www.mavetju.org/unix/netrc.php ''~/.netrc'' file]:

* Create/edit your ''~/.vimrc'' to switch to ASCII mode on start, add:
let g:netrw_ftpmode="ascii"

* Create/edit your ''~/.netrc'' to have credentials ready, add:
machine my-machine-name
login myuser
password mypassword
macdef init
ascii

Important: The file '''must'''
* end with a blank line,
* set <code>chmod 600 ~/.netrc</code>, otherwise the ''ftp'' command refuses to use it.

* Edit files giving vim the remote URL:
vim <nowiki>ftp://my-machine-name/mylib/mysrcpf.mymember</nowiki>

With this toolset, you can develop with a tall ''vim'' window on your desktop machine, klick on a 5250-session to compile and maybe have a second 5250 session open for running the program.
<gallery class="center">
File:Vim-screen-dspf-dds.png|Vim showing 91 lines of DDS
File:Vim-screen-rpgle.png|Vim showing 91 lines of RPG IV positional code
</gallery>

== Preparation ==
As you probably know from your experience with common systems, it's considered bad habit to work with administrative rights all of the time. I recommend [[Creating a User Account]] for you to properly work without accidentally doing harm to the system. ''Sign in'' as this user now. This gives you a subset of menu items in the ''main'' menu and makes your own Library your ''current library''. For now, consider this almost the same thing as using the <code>cd</code> command of a shell to make a directory your current one.

As outlined in [[Filesystems and Files on the AS/400#File Types|File Types]], we'll need to create a file to hold source code. Let's do that and make the maximum line length a bit larger for more flexibility.<ref>IBM delivers ready made, empty source files in the QGPL-Library. Every class of file has it's own source file, such as QCLSRC for CL source code. I consider this confusing: Working on multiple files is more cumbersome than working on multiple members of one file. The files in a member could be marked as a given type of content. The only drawback is that one must use distinct names on members. The QSYS.LIB-Filesystem can hold files with the same ''name'' as long as they have distinct ''types''.</ref>
CRTSRCPF FILE(SOURCES) RCDLEN(112)

The white text in the status line will show you that the file has been created.

== Working with Members in Files ==
The easiest way to edit members of source files is to utilize part of the ''Program Development Manager''.
WRKMBRPDM FILE(SOURCES)

You'll be presented with a screen form to handle members. Press <code>F6</code> to show the ''Start SEU''<ref>Source Entry Utility.</ref> screen. Fill out the two first fields:
* SRCMBR is ''addrpf'' (for addresses physical file),
* TYPE is ''pf'' for physical file.

Press <code>Enter</code> to start SEU and present an empty file.

== Coding the Database Table ==
We're going to code the description of a database file, that is the traditional way to create database files in OS/400.<ref>Database files can also be created with the interactive SQL facility, though. But one can't derive a textual description from a SQL created database file.</ref> The ''language'' for the statements is called ''DDS''<ref>Data Description Statements.</ref> that is not a line oriented description like HTML. The position of statements within one line are also important.<ref>This stems from old times when punch cards were used for programming stuff. Since compatibility is mandatory, IBM extended DDS from time to time with new keywords, but left the previous stuff alone.</ref>

For an address book, let's assume the following fields are of primary interest:
* Name,
* First Name,
* Birthday,
* Street and Number,
* ZIP Code and City Name,
* Phone Number,
* Cellular Phone Number,
* E-Mail Address.

Plain DDS Code looks like this:
A UNIQUE
A R ADDRTBL
A NAME 48A
A FIRSTNAME 48A
A BIRTHDAY L
A STREETNUM 48A
A ZIPCITY 48A
A PHONE 48A ALWNULL
A CELLPHONE 48A ALWNULL
A EMAIL 48A ALWNULL
A K NAME
A K FIRSTNAME
A K BIRTHDAY

To understand a bit better, I'll elaborate.

As you can see, everything is written in upper case. To be honest, I don't know if a file with lower case characters in any place but comments is compilable.

Every line starts with an ''A'' in the 6th place of a line. The first five once were reserved for optional line numbering but I've seen no examples where this is actually done.

The first line denotes that this file may not have duplicate values in key fields (coded later).

The second line names this ''record format''. This should be different from the file name. If not, you'll earn compiler warnings: In the actual program, when working with that file, it's ambiguous if you refer to the file name or the record format. For that reason, I name record formats for physical files with the suffix ''tbl''. 
Physical Files must have exactly one record format.

The next few lines define the fields of the database file. I don't want getting unnecessary initial complexity, so I defined all fields as ch'''a'''racter with static 48 places.<ref>The AS/400 database knows about ''varchar'' field, but the handling in the code later would be really clumsy. So I refrain from that for now. Not used places are automatically filled with blanks.</ref> The ''Birthday''-field is the only exception, that is defined as a date field.

Some fields may be left empty. To save space, these may be NULL, that is not the same as ''empty''. Therefore they are defined with an additional parameter ''alwnull''. 
NULL fields permit fields of records to utilize no space at all, but need special attention in the code. This isn't as clumsy as coping with ''varlen'' strings, though.

The last three lines denote ''key fields''. When reading records, they'll be retrieved in key order (ascending by default). According to the first line, entries must be ''unique'' over all key fields. So you may not have more than one entry named Fred Flintstone born on 1989-08-20. This is a somewhat crude way to prevent duplicates.<ref>If Fred has more than one place to live, the right way to handle that would be to create a second table that lists every possible street address for a given Tuple Name—First Name—Birthday. Of course we might introduce entry IDs to keep the address table small. As said, I want to keep complexity low.</ref>

After saving the member, one may type ''14'' into the field of the ''wrkmbrpdm'' members list to instruct it to compile the file with default values to the default location that is your current library.

=== A Word about the Compilation of Objects ===
Compilation per default takes place as background (batch) job that is more friendly to other tasks running in interactive mode but it's a bit clumsy to handle. One needs to look into the session's [[Messages and Message Queues|message queue]] to see if the compilation succeeded or failed, to be followed by a look into the [[Spooled Files and Output Queues|spooled file]], searching the verbose compiler output for possible culprits. 
One may press <code>F18</code> in the ''wrkmbrpdm'' screen to access another screen with user based defaults. The third option must be set to ''N''.

== See also ==
* [[Editing Source Files using Eclipse and FTP]]

== Weblinks ==
* [https://www.pocnet.net/computer/fallout/ Helper Programs] for the Fallout Role Playing Game Series
* [https://github.com/PoC-dev/as400-sfltemplates Templates] for easy development of one-database-table maintenance programs

== Footnotes ==
<references />

[[Category: Programming]]

Filesystems and Files on the AS/400

2026-03-18T17:58:04Z

Greenfive: fixed typo, fixed grammar

{{FIXME|
* Add Chapter how to create physical and logical files,
* Move explanations to access paths to a programming article?}}
Since the AS/400 has been designed as a business helper machine, its processing capabilities mainly cover structured data with comparatively small record sizes. With the ongoing evolution of OS/400, more functionality was added by IBM that weakens this concise definition, though. On the other hand, the same added functionality improves ease of data storage for, and data exchange with other platforms.

== Different approaches of data storage and processing ==
Structured data can be processed most efficiently when the underlying logical and physical facilities support this approach natively instead of mimicking.

By comparison, most platforms outside the world of mainframes and minicomputers, as they were once known, operate on byte streams. Byte streams don't have any natural structure visible from outside. That is, without analyzing the content of the stream at any given moment, there is no chance to map the current byte position into any logical structure. 
This is somewhat similar to the differences between data transmission in an asynchronous manner (such as the RS-232 protocol specifies) versus synchronous. With asynchronous transfers, data payload has to be enriched with additional information to enable processing logic to chop the continuous stream of data into pieces to enable processing facilities to work on them in a structured way.

Most, if not all data is based on some structure, though. The simplest structure is a blob of data of a given size, usually named a ''block''. When looking closely, conversion from, and to byte streams from and to blocks takes place on many levels:
* Storage of data on hard disks or in solid state memory in blocks of 512 Bytes or 4 [[wikipedia:Kibibyte|KiB]],
* Paging and Swapping activity of virtual memory systems also takes place in blocks instead of streams,
* Data Transfer over todays networks is done in pieces called ''frames''.<ref>To be precise, this isn't done to ease or enable record oriented processing but to enable sharing of physical media (like cables) between many participants of the network.</ref>

On a higher level of a file's content definition, more elaborated structuring of stream-based data can be found:
* Pixel-oriented picture files include a header that dynamically defines the format of the data that follows,
* Movie files are defined by different kinds of data blocks: To save space, many frames are only saved as differences to the picture before,<ref>From time to time, a key frame contains a complete picture to compensate for data loss, that can happen for many reasons.</ref>
* Todays most used text processing software internally uses [[wikipedia:XML|XML]] to give seemingly unstructured data a structure,
* Application Program Files also are structured in a way that permits the operating system to efficiently load and execute the contained code,
* Even plain text files have a structure: They contain of words, sentences, and paragraphs.

The mapping of structured data into byte streams leverages processing speed with flexibility to store arbitrary complex data structures into very simple data structures on disk. Today, the processing overhead of mapping byte streams into records (blocks) or more elaborated data structures is almost nullified by cheap availability of sheer processing power.

=== Storage Overhead ===
However, mapping byte streams into logical blocks of information often creates ''storage overhead''. For example, a plain text file with the word 'Hello' uses only 5 Bytes of actual data<ref>Counted as ASCII, or SBCS as named in the AS/400-World.</ref> (sometimes followed by implicit end of line marker(s)), the file occupies at least 512 Bytes data, since this is the minimum block size on usual hard disks. If the file is expanded to exactly 512 Bytes by adding more text, the overhead is zero. Adding just one character starts the continuation of data in the next disk block and again occupies the whole block of that 511 Bytes are wasted. 
This overhead might be negligible when looking at todays file sizes. But, every OS utilizes configuration files, that usually are small and fit the above description perfectly. ''/etc/hosts'' may be 680 Bytes long on a given system, that makes up 344 Bytes wasted space, according to the above calculation.<ref>Actually, with todays partition sizes, Linux uses 4 KiB blocks, not 512 Bytes. This raises the overhead of said ''/etc/hosts'' to 3416 Bytes.</ref>

Today, this storage overhead is also almost nullified by cheap availability of gigantic hard disk storage capacity.

=== Latency ===
Considering the above description, it's easily observable that actual data is scattered across a greater area of a given hard disk with many holes in between, compared to all files concatenated to one big blob. This increases latency, because the disk's head assembly must move more often to access a certain block on the physical media.

Today, I/O latencies are nullified by cheap availability of solid state memory “disks”.

== AS/400 IFS ==
Early incarnations of the AS/400 had no facilities to save or process byte-based ''stream files''. There only was the possibility to create files with a fixed record structure. With the introduction of the ''integrated file system'' in OS/400 Version 3, saving and processing of byte stream files has been made possible. The classical record oriented file system was in turn named ''QSYS.LIB''-file system and looks like a classical Unix mount point, when looking with applications being aware of the IFS, such as ''qsh'', the minimal UNIX like shell. The rest of the filesystems allow to store stream files. 
Up to today, ''QSYS.LIB'' is still the most important file system on AS/400's because most part of the OS is stored there, as well as configuration database files and, of course application specific database files.

This ''IFS'' is not the same as file systems in Linux or other common operating systems. There, one or more disks are ''partitioned'': Space from one disk split into slices of usable space by software. To make use of the space available in this partitions, a ''file system'' has to be created. That is, database-like structures for the mapping of files to disk blocks are written into any given partition space. The space available for the creation of files on any given file system is thus fixed to the size of the partition, minus overhead, minus space needed for the aforementioned data structures. Multiple filesystems have been conceived over decades with a determined set of features, so it's possible to use different file systems to match the different storage needs of different types of files.

On the other side, the IFS of the AS/400 is a purely ''logical structure'' that allocates disk blocks as requested by the operating system. If a record oriented file in the classical ''QSYS.LIB''-filesystem is filled with data, disk blocks are filled with this data and are thus no longer available for storage. If someone transfers a picture file to the AS/400's ''/QOpenSys''-Filesystem, disk-blocks are allocated for that data and also cannot be allocated for storage. There is no static assignment for these different file systems to partitions or disk blocks.

'''Stuff saved in file systems on the AS/400 is usually referred to as ''objects''.

== AS/400 files ==
The AS/400 file handling shares a common heritage of file handling found in IBM mainframes (MVS or z/OS partitioned data sets). An AS/400 ''file'' defines at least the record length. All ''members'' of these file share this maximum length and thus are constrained to this length. 
In addition, an ''externally defined file'' can be more strict than an MVS PDS, by not only enforcing a record length, but also an actual field layout and -content over this record. <ref>The term ''external'' is greatly misleading here, because the file definition is ''external'' to the ''program'' using that file. Indeed, the field layout is ''internal'' to the file object.</ref>
This can be roughly compared to a directory on Linux or (better) a tar file, where one may only add TIFF image data with a common horizontal pixel count.<ref>This is fairly unpractical in computing as we know it.</ref>

For example, by creating a ''source file'' on an AS/400, one defines a ''file'' with a certain record length.<ref>92 Bytes by default.</ref> One may use this ''file'', create ''members'' and add program source code to these members at will, but it's not possible to add data with any line longer than 92 characters into any member of that file.

=== File Types ===
Basically there are four types of files found in any AS/400's classical ''QSYS.LIB'' filesystem:
* '''Source Files''' are ''physical files'', aka ''database files'' with at least two fields per record:
** Line of Text, primarily to hold program code to be compiled,
** Timestamp, records the last change of that particular line.<ref>SEU shows these timestamps on the very far right of it's display.</ref>
* '''Physical Files''' are again database files but with a user defined record length and field format.
* '''Logical Files''' are files that reference one or more physical files, but do not contain any data. LFs can be compared to the SQL terms:
** ''view'': Show only a subset of fields of PF,
** ''join'': Show a superset of fields of more than one PF,
** ''index'': Apply different indexes, and thus record sorting.
: In addition, restrictions about which records to select or omit from the PF can be added.
* '''Other Files''' that appear as type ''*FILE'' in output listings that include, but are not limited to
** ''Display Files'' that map data to positions on the screen and enable user interaction to this forms,
** ''Printer Files'' that map data to positions on a virtual sheet of paper for later printer output,
** ''Inter Communication Facility Files'' who provide an easy way for application programs to talk to other programs on the same or remote AS/400's,
** ''Save files'' that provide roughly the same functionality as tar files on Linux, enabling bundling of files (objects) of different types into one self-contained archive, for easier transmission to other systems for re-extracting or permanent storage.

Handling of these files will be explained in separate articles.

==== A word on physical and logical (database) files ====
The main usage of logical files is to provide different ''access paths'' compared to the linked physical file to read data. An access path is just one or more [[wikipedia:Database index|database index(es)]] created for one or more fields of a given physical file.

When accessing database files through the classical READ(P)(E)-API-Calls, there's no possibility to sort a result set dynamically as it's easily done with [[wikipedia:SQL|SQL]]. There's not even a result set. There's just kind of a ''file pointer'' that points to the last read row, and the order in that the rows are retrieved by subsequent READs (that increment this file pointer) are determined by the access path(s). 
Keep in mind that this handling of structured data stems from a time period when computers had only a tiny fraction of today's processing power<ref>Processing as in ''Read from disk — Modify — Write to disk''.</ref> and storage capacity.

Considering the clear positioning of the AS/400 as ''database machine'' by IBM, and adding the fact that the precursor to SQL was created in the early 1970s by IBM employees, SQL might have been available on the AS/400 from it's first release. 
On older machines, SQL proves to be painfully slow, especially when dealing with huge database tables. Since the SQL engine dynamically adds database indexes as needed when processing queries, getting data out of a table can be really slow when such a temporary index must be created in advance.

On the other side, utilizing the legacy API-Calls to retrieve and process random records with CHAIN-READ from multi-GiB-sized database tables is surprisingly quick on old machines for many reasons.

This somewhat restricted API call interface explains why data handling within applications on the AS/400 feel so fundamentally different compared to what we're used to.

Most common platforms utilize SQL:
* The user is providing (part of) a query string, the database picks matching rows and
* the application delivers a limited view of the whole database content back to the user.
** The user may easily change sorting of the result records to be based on any single field by usually clicking on the respective table header field in the application's UI,
* the user may finally select the desired record for further processing.

AS/400 applications usually…
* provide a complete list view<ref>These lists are called ''subfiles''.</ref> of the entire database content, with a fixed sort order. The user may either scroll to the desired record, or type part of a search string in a designated input line and press <code>Enter</code>.
* The application reloads the list view after CHAINing (placing the ''file pointer'') to the first record matching (part of) the search string<ref>To be precise, the search string is matched against the '''start''' of the respective strings in the key field. It's not a <code>… LIKE '%FOOBAR%'</code>-matching that is simply not possible with the legacy API.</ref>. The rest of the list is filled with subsequent READs until the list is full or [[wikipedia:End-of-file|EOF]] is reached.
** Skillful programming techniques allow the user to freely scroll forward and backward from the record retrieved in the last step.
* The user may finally select the desired record for further processing.

Programs don't need to maintain the list programmatically xor allocate memory for loading the whole database. Details will be explained in the articles about programming.

== Footnotes ==
<references />

[[Category: Basic Knowledge]]

5250 colors and display attributes

2025-07-09T13:55:34Z

Greenfive: fixed typo

The IBM '''5250''' data stream was devised in the 1970's. Color was not thought of, even if color displays were well known for a long time from TV. After the later introduction of color displays for computer terminal screens, already established '''display attributes''' were reused for indicating which color something on the screen should have.

Display attributes are encoded in a single, otherwise invisible screen position directly preceding the screen element(s) the attribute is applied to.<ref>The DDS logic takes care to switch off set attributes after a screen element's end.</ref> This ''attribute byte'' can be modified with the DDS keywords <tt>COLOR</tt> and <tt>DSPATR</tt> and indicators. Since indicators are limited in count, conditionally applying many attributes to many screen objects can quickly exhaust the number of available indicators for a given screen (record format).<ref>It should be noted that in a subfile, each record has an independent set of indicators.</ref>

== Attributes ==
The basic display attributes are specified as follows. Counted from right to left (LSB to MSB)!
{|class="wikitable"
!Bit
!Attribute (DSPATR-Name)
|-
|align="right"|0
|Reverse Image (RI)
|-
|align="right"|1
|High Intensity (HI)
|-
|align="right"|2
|Underline (UL)
|-
|align="right"|3
|Blink (BL)
|-
|align="right"|4
|Column separator (CS)
|-
|align="right"|5
|Always 1
|-
|align="right"|6
|Always 0
|-
|align="right"|7
|Protect (PR)
|}

== Attribute to color mapping ==
As stated, color was introduced later. Hence, display attribute bits have been reused for setting color:
* Blink = Red
* Column separator = Turquoise (cyan)
* High intensity = White

Further colors can be displayed by combining bits 1, 3, and 4:
* Blink, column separator = Pink (magenta)
* Blink, column separator, high intensity = Blue
* Column separator, high intensity = Yellow

Normally, you don't need to memorize all of that, but just use the DDS <tt>COLOR(xxx)</tt> keyword to enable the available colors:
{|class="wikitable"
!Desired color
!DDS <tt>COLOR</tt> statement
!equivalent DDS <tt>DSPATR</tt> statement
|-
|Black
| 
|<tt>DSPATR(ND)</tt>
|-
|White
|<tt>COLOR(WHT)
|<tt>DSPATR(HI)</tt>
|-
|Red
|<tt>COLOR(RED)</tt>
|<tt>DSPATR(BL)</tt>
|-
|Green
|<tt>COLOR(GRN)</tt>
| 
|-
|Blue
|<tt>COLOR(BLU)</tt>
|<tt>DSPATR(BL CS HI)</tt>
|-
|Yellow
|<tt>COLOR(YLW)</tt>
|<tt>DSPATR(CS HI)</tt>
|-
|Pink (magenta)
|<tt>COLOR(PNK)</tt>
|<tt>DSPATR(BL CS)</tt>
|-
|Turquoise (cyan)
|<tt>COLOR(TRQ)</tt>
|<tt>DSPATR(CS)</tt>
|}

All but black can be combined with the Reverse Video (<tt>RI</tt>) attribute to have black letters on colored background.<ref>This should be used sparsely, because it's very eye straining.</ref>

Depending on the settings of the used color display terminal or emulator, attributes and color might be shown concurrently. This is most apparent when setting the color of an input capable field to cyan, and the result not only is cyan, but also ''looks ugly'', due to column separators being displayed. Most display terminals or emulators have a configuration setting to only show colors and not apply attributes.<ref>Due to reusing of attribute bits, changing this setting disables the display of attributes globally. You can't then display one field with e. g. column separators. Instead, the field is then colored cyan.</ref>

== Table of valid attribute byte values ==
{|class="wikitable"
!Hex
!Binary
!Color
!Attribute(s)
!Screen appearance
|-
|0x20
|0b00100000
|GRN
| 
|Green
|-
|0x21
|0b00100001
|GRN
|RI
|Green, reverse image
|-
|0x22
|0b00100010
|WHT
| 
|White
|-
|0x23
|0b00100011
|WHT
|RI
|White, reverse image
|-
|0x24
|0b00100100
|GRN
|UL
|Green, underline
|-
|0x25
|0b00100101
|GRN
|UL RI
|Green, underline, reverse image
|-
|0x26
|0b00100110
|WHT
|UL
|White, underline
|-
|0x27
|0b00100111
| 
|ND
|Nondisplay
|-
|0x28
|0b00101000
|RED
| 
|Red
|-
|0x29
|0b00101001
|RED
|RI
|Red, reverse image
|-
|0x2a
|0b00101010
|RED
|HI
|Red, high intensity
|-
|0x2b
|0b00101011
|RED
|HI RI
|Red, reverse image, high intensity
|-
|0x2c
|0b00101100
|RED
|UL
|Red, underline
|-
|0x2d
|0b00101101
|RED
|UL RI
|Red, unline, reverse image
|-
|0x2e
|0b00101110
|RED
|UL BL
|Red, underline, blink
|-
|0x2f
|0b00101111
|RED
|ND
|Nondisplay
|-
|0x30
|0b00110000
|TRQ
|CS
|Turquoise, column separator
|-
|0x31
|0b00110001
|TRQ
|CS RI
|Turquoise, column separator, reverse image
|-
|0x32
|0b00110010
|YLW
|CS
|Yellow, column separator
|-
|0x33
|0b00110011
|WHT
|RI CS
|White, reverse image, column separator
|-
|0x34
|0b00110100
|TRQ
|UL CS
|Turquoise, underline, column separator
|-
|0x35
|0b00110101
|TRQ
|UL RI CS
|Turquoise, underline, reverse image, column separator
|-
|0x36
|0b00110110
|YLW
|UL CS
|Yellow, underline, column separator
|-
|0x37
|0b00110111
| 
|ND
|Nondisplay
|-
|0x38
|0b00111000
|PNK
| 
|Pink
|-
|0x39
|0b00111001
|PNK
|RI
|Pink, reverse image
|-
|0x3a
|0b00111010
|BLU
| 
|Blue
|-
|0x3b
|0b00111011
|BLU
|RI
|Blue, reverse image
|-
|0x3c
|0b00111100
|PNK
|UL
|Pink, underline
|-
|0x3d
|0b00111101
|PNK
|UL RI
|Pink, underline, reverse image
|-
|0x3e
|0b00111110
|BLU
|UL
|Blue, underline
|-
|0x3f
|0b00111111
| 
|ND
|Nondisplay
|-
|0xa0
|0b10100000
|GRN
|PR
|Green, protected
|-
|0xa1
|0b10100001
|GRN
|PR RI
|Green, protected, reverse image
|-
|0xa2
|0b10100010
|WHT
|PR
|White, protected
|-
|0xa3
|0b10100011
|WHT
|PR RI
|White, protected, reverse image
|-
|0xa4
|0b10100100
|GRN
|PR UL
|Green, protected, underline
|-
|0xa5
|0b10100101
|GRN
|PR UL RI
|Green, protected, underline, reverse image
|-
|0xa6
|0b10100110
|WHT
|PR UL
|White, protected, underline
|-
|0xa7
|0b10100111
| 
|ND PR
|Nondisplay, protected
|-
|0xa8
|0b10101000
|RED
|PR
|Red, protected
|-
|0xa9
|0b10101001
|RED
|PR RI
|Red, protected, reverse image
|-
|0xaa
|0b10101010
|RED
|PR HI
|Red, protected, high intensity
|-
|0xab
|0b10101011
|RED
|PR HI RI
|Red, protected, reverse image, high intensity
|-
|0xac
|0b10101100
|RED
|PR UL
|Red, protected, underline
|-
|0xad
|0b10101101
|RED
|PR UL RI
|Red, protected, unline, reverse image
|-
|0xae
|0b10101110
|RED
|PR UL BL
|Red, protected, underline, blink
|-
|0xaf
|0b10101111
| 
|ND PR
|Nondisplay, protected
|-
|0xb0
|0b10110000
|TRQ
|PR CS
|Turquoise, protected, column separator
|-
|0xb1
|0b10110001
|TRQ
|PR CS RI
|Turquoise, protected, column separator, reverse image
|-
|0xb2
|0b10110010
|YLW
|PR CS
|Yellow, protected, column separator
|-
|0xb3
|0b10110011
|WHT
|PR RI CS
|White, protected, reverse image, column separator
|-
|0xb4
|0b10110100
|TRQ
|PR UL CS
|Turquoise, protected, underline, column separator
|-
|0xb5
|0b10110101
|TRQ
|PR UL RI CS
|Turquoise, protected, underline, reverse image, column separator
|-
|0xb6
|0b10110110
|YLW
|PR UL CS
|Yellow, protected, underline, column separator
|-
|0xb7
|0b10110111
| 
|ND PR
|Nondisplay, protected
|-
|0xb8
|0b10111000
|PNK
|PR
|Pink, protected
|-
|0xb9
|0b10111001
|PNK
|PR RI
|Pink, protected, reverse image
|-
|0xba
|0b10111010
|BLU
|PR
|Blue, protected
|-
|0xbb
|0b10111011
|BLU
|PR RI
|Blue, protected, reverse image
|-
|0xbc
|0b10111100
|PNK
|PR UL
|Pink, protected, underline
|-
|0xbd
|0b10111101
|PNK
|PR UL RI
|Pink, protected, underline, reverse image
|-
|0xbe
|0b10111110
|BLU
|PR UL
|Blue, protected, underline
|-
|0xbf
|0b10111111
| 
|ND PR
|Nondisplay, protected
|}

=== Arithmetics of attributes ===
Because the meaning of attributes or color is well defined, many lines of <tt>IF</tt> or <tt>SELECT</tt> statements in RPG can possibly be substituted by simple arithmetics, like additions and bit operations.

Examples:

Base value is always 0x20.
* To have a field displayed in reverse image, add 1,
* to have a field displayed in red, add 8,
* etc.

== Weblinks ==
* [https://www.ibm.com/docs/en/i/7.6.0?topic=80-color-color-keyword-display-files COLOR (Color) keyword for display files], IBM DDS reference
* [https://www.ibm.com/docs/en/i/7.6.0?topic=80-dspatr-display-attribute-keyword-display-files DSPATR (Display Attribute) keyword for display files], IBM DDS reference
* [[wikipedia:IBM 5250|IBM 5250]], Wikipedia
* [https://www.rpgpgm.com/2013/10/condition-display-fields-without.html Condition display fields without indicators], rpmpgm.com

== Footnotes ==
<references />

[[Category: Programming]]
[[Category: System Internals]]

How to program an Application

2024-02-08T18:44:17Z

Greenfive: fixed typos

{{FIXME|Finish article.}}
Getting started on programming the AS/400 isn't hard. According to Frank Soltis, the most effective way to get a machine sold is to actually bring an AS/400 along to the future customer, and show an C''x''O in a one hour crash course how to code a simple ''business application''. 
Well, something an arbitrary business buffoon can do, you can do, too. Since the usual [[wikipedia:"Hello, World!" program|Hello World]] is just too lame, let's start with an actually usable example: A simple address book application.

== Components of a typical Program ==
Since we're talking about a database machine, the first we need to know is how to create ''Database Tables''.

Editing Database Content is usually done in screen ''forms''. These are called ''Display Files''. So, let's also learn how to create Display Files in the next turn.

Finally, there must be a program for handling key presses from the user and coping with the I/O of database- and display files.

Optionally, there can be more files to support additional functions in a program. Printer files allow for formatted output (tabular report) to a printer. InterSystem Communication Files allow for Record-I/O to other programs on the local or a remote host. We will not dig into these, for now.

=== Programming Languages ===
Available on a full install of an older OS/400 are compilers for
* [[wikipedia:C (programming language)|C]]/[[wikipedia:C++|C++]],
* [[wikipedia:COBOL|COBOL]],
* [[wikipedia:IBM i Control Language|CL]], the equivalent of batch/shell scripts,
* [[wikipedia:Java (programming language)|Java]],
* [[wikipedia:IBM RPG|RPG]],<ref>Report Program Generator.</ref>

and interpreters for
* [[wikipedia:Rexx|REXX]], a scripting language derived from the mainframe world.

RPG and COBOL are available in two flavors: [[wikipedia:Integrated Language Environment|ILE]] and [https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_73/ilec/opmdesc.htm OPM]. See linked explanations for details.

It makes no sense to utilize OPM compilers for new projects. It's best you save space and deinstall these as well as the ''*PRV'' variants, that generate code for the immediately previous OS release. We'll solely focus on ILE since it has (amongst others) the advantage of linking (binding) compiled object code from different languages together to form a program.

The IBM website features [https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzahg/rzahglanguages.htm extensive documentation] for each language.

== A Word on Editors ==
If you are familiar with ''vi'' or ''vim'' on Unix-like platforms, learning SEU will be an easy task. You may alternatively utilize an editor of your choice on the platform of your choice.<ref>Windows Notepad is '''not''' a proper editor. It's sufficient for notes, though. Hence the name.</ref> To help you chose, we'll take a closer look on the pros and cons of each solution.

{|class="wikitable"
!Editor
!Pro
!Con
|-
|valign="top"|'''SEU on OS/400'''
|valign="top"|Built-in contextual support for many file types and kind of lines with precise column requirements. Editing is done in an implicit array of blanks in permanent overwrite mode for easier placement of characters in proper position.
|valign="top"|Can display only a few lines of code, that reduces clarity and thus forces a lot of scrolling. Uniform colouring of code is increasingly uncommon and makes it even more difficult to keep an overview.
|-
|valign="top"|'''Other on other platforms'''
|valign="top"|Familiar operation and UI. Easy content duplication with clearly visible text range selection, and copy-paste. Possibility to create custom rules for syntax highlighting.
|valign="top"|Cumbersome handling: After every change that should result in a compile run, changed files have to be transferred back in some way.
|}

The Killer-Feature of SEU is the extensive contextual support it provides for RPG and DDS, and partly for COBOL. For beginners, it's much easier to utilize this helpful function often until some knowledge, routine and working programs as copy-source have been established.

=== Vim ===
Vim is most often the editor of choice for seasoned Text-UI affictionados. Since ''vim'' is able to open files on remote-locations via URL notation, you can perfectly use it for editing on the fly from any platform vim is available for. There are even extensions for [https://github.com/kaimoku/vim-syntax-for-i syntax-highlighting] of common code.<ref>No longer working with vim 8.</ref> Automation of login can be done via [http://www.mavetju.org/unix/netrc.php ''~/.netrc'' file]:

* Create/edit your ''~/.vimrc'' to switch to ASCII mode on start, add:
let g:netrw_ftpmode="ascii"

* Create/edit your ''~/.netrc'' to have credentials ready, add:
machine my-machine-name
login myuser
password mypassword
macdef init
ascii

Important: The file '''must''' end with a blank line! Do a <code>chmod 600 ~/.netrc</code> for some more security.

* Edit files giving vim the remote URL:
vim <nowiki>ftp://my-machine-name/mylib/mysrcpf.mymember</nowiki>

With this toolset, you can develop with a tall ''vim'' window on your desktop machine, klick on a 5250-session to compile and maybe have a second 5250 session open for running the program.
<gallery class="center">
File:Vim-screen-dspf-dds.png|Vim showing 91 lines of DDS
File:Vim-screen-rpgle.png|Vim showing 91 lines of RPG IV positional code
</gallery>

== Preparation ==
As you probably know from your experience with common systems, it's considered bad habit to work with administrative rights all of the time. I recommend [[Creating a User Account]] for you to properly work without accidentally doing harm to the system. ''Sign in'' as this user now. This gives you a subset of menu items in the ''main'' menu and makes your own Library your ''current library''. For now, consider this almost the same thing as using the <code>cd</code> command of a shell to make a directory your current one.

As outlined in [[Filesystems and Files on the AS/400#File Types|File Types]], we'll need to create a file to hold source code. Let's do that and make the maximum line length a bit larger for more flexibility.<ref>IBM delivers ready made, empty source files in the QGPL-Library. Every class of file has it's own source file, such as QCLSRC for CL source code. I consider this confusing: Working on multiple files is more cumbersome than working on multiple members of one file. The files in a member could be marked as a given type of content. The only drawback is that one must use distinct names on members. The QSYS.LIB-Filesystem can hold files with the same ''name'' as long as they have distinct ''types''.</ref>
CRTSRCPF FILE(SOURCES) RCDLEN(112)

The white text in the status line will show you that the file has been created.

== Working with Members in Files ==
The easiest way to edit members of source files is to utilize part of the ''Program Development Manager''.
WRKMBRPDM FILE(SOURCES)

You'll be presented with a screen form to handle members. Press <code>F6</code> to show the ''Start SEU''<ref>Source Entry Utility.</ref> screen. Fill out the two first fields:
* SRCMBR is ''addrpf'' (for addresses physical file),
* TYPE is ''pf'' for physical file.

Press <code>Enter</code> to start SEU and present an empty file.

== Coding the Database Table ==
We're going to code the description of a database file, that is the traditional way to create database files in OS/400.<ref>Database files can also be created with the interactive SQL facility, though. But one can't derive a textual description from a SQL created database file.</ref> The ''language'' for the statements is called ''DDS''<ref>Data Description Statements.</ref> that is not a line oriented description like HTML. The position of statements within one line are also important.<ref>This stems from old times when punch cards were used for programming stuff. Since compatibility is mandatory, IBM extended DDS from time to time with new keywords, but left the previous stuff alone.</ref>

For an address book, let's assume the following fields are of primary interest:
* Name,
* First Name,
* Birthday,
* Street and Number,
* ZIP Code and City Name,
* Phone Number,
* Cellular Phone Number,
* E-Mail Address.

Plain DDS Code looks like this:
A UNIQUE
A R ADDRTBL
A NAME 48A
A FIRSTNAME 48A
A BIRTHDAY L
A STREETNUM 48A
A ZIPCITY 48A
A PHONE 48A ALWNULL
A CELLPHONE 48A ALWNULL
A EMAIL 48A ALWNULL
A K NAME
A K FIRSTNAME
A K BIRTHDAY

To understand a bit better, I'll elaborate.

As you can see, everything is written in upper case. To be honest, I don't know if a file with lower case characters in any place but comments is compilable.

Every line starts with an ''A'' in the 6th place of a line. The first five once were reserved for optional line numbering but I've seen no examples where this is actually done.

The first line denotes that this file may not have duplicate values in key fields (coded later).

The second line names this ''record format''. This should be different from the file name. If not, you'll earn compiler warnings: In the actual program, when working with that file, it's ambiguous if you refer to the file name or the record format. For that reason, I name record formats for physical files with the suffix ''tbl''. 
Physical Files must have exactly one record format.

The next few lines define the fields of the database file. I don't want getting unnecessary initial complexity, so I defined all fields as ch'''a'''racter with static 48 places.<ref>The AS/400 database knows about ''varchar'' field, but the handling in the code later would be really clumsy. So I refrain from that for now. Not used places are automatically filled with blanks.</ref> The ''Birthday''-field is the only exception, that is defined as a date field.

Some fields may be left empty. To save space, these may be NULL, that is not the same as ''empty''. Therefore they are defined with an additional parameter ''alwnull''. 
NULL fields permit fields of records to utilize no space at all, but need special attention in the code. This isn't as clumsy as coping with ''varlen'' strings, though.

The last three lines denote ''key fields''. When reading records, they'll be retrieved in key order (ascending by default). According to the first line, entries must be ''unique'' over all key fields. So you may not have more than one entry named Fred Flintstone born on 1989-08-20. This is a somewhat crude way to prevent duplicates.<ref>If Fred has more than one place to live, the right way to handle that would be to create a second table that lists every possible street address for a given Tuple Name—First Name—Birthday. Of course we might introduce entry IDs to keep the address table small. As said, I want to keep complexity low.</ref>

After saving the member, one may type ''14'' into the field of the ''wrkmbrpdm'' members list to instruct it to compile the file with default values to the default location that is your current library.

=== A Word about the Compilation of Objects ===
Compilation per default takes place as background (batch) job that is more friendly to other tasks running in interactive mode but it's a bit clumsy to handle. One needs to look into the session's [[Messages and Message Queues|message queue]] to see if the compilation succeeded or failed, to be followed by a look into the [[Spooled Files and Output Queues|spooled file]], searching the verbose compiler output for possible culprits. 
One may press <code>F18</code> in the ''wrkmbrpdm'' screen to access another screen with user based defaults. The third option must be set to ''N''.

== See also ==
* [[Editing Source Files using Eclipse and FTP]]

== Weblinks ==
* [https://www.pocnet.net/computer/fallout/ Helper Programs] for the Fallout Role Playing Game Series
* [https://github.com/PoC-dev/as400-sfltemplates Templates] for easy development of one-database-table maintenance programs

== Footnotes ==
<references />

[[Category: Programming]]