Getting Data in and out of your AS/400: Difference between revisions
(→Communication options of the AS/400: Reworked) |
(De-whichified) |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 18: | Line 18: | ||
Support for inter-vendor communications was provided with the licensed product ''TCP/IP Connectivity Utilities for AS/400'', [https://books.google.de/books?id=DR0EAAAAMBAJ&pg=PA1#v=onepage&q&f=false introduced by IBM in 1990]. | Support for inter-vendor communications was provided with the licensed product ''TCP/IP Connectivity Utilities for AS/400'', [https://books.google.de/books?id=DR0EAAAAMBAJ&pg=PA1#v=onepage&q&f=false introduced by IBM in 1990]. | ||
Today, IP on the AS/400 allows a multitude of communication tasks utilizing IP instead of SNA, | Today, IP on the AS/400 allows a multitude of communication tasks utilizing IP instead of SNA, that is way better understood by newbies in the IBM world of networking, then SNA. | ||
Even though that IBM moved some IP processing into the corresponding I/O processor(s), IP transfers have a comparatively high impact on CPU load. Slower machines like the 9401-150 benefit greatly from utilizing SNA for bulk data transfers, since SNA hogs about one third of the CPU compared to IP. | Even though that IBM moved some IP processing into the corresponding I/O processor(s), IP transfers have a comparatively high impact on CPU load. Slower machines like the 9401-150 benefit greatly from utilizing SNA for bulk data transfers, since SNA hogs about one third of the CPU compared to IP. | ||
| Line 30: | Line 30: | ||
* IPX. | * IPX. | ||
It is not clear if the ''NetServer'' feature<ref>Basically a piece of software | It is not clear if the ''NetServer'' feature<ref>Basically a piece of software that provides NetBIOS-Services: File Sharing and Printing. Kind of ''Samba from IBM''.</ref> also provides support for [[wikipedia:NetBIOS Frames protocol|NetBIOS Frames protocol]], and that releases of OS/400 support this, if any. | ||
Additionally with the so called ''AnyNet''-feature, OS/400 allows any of the above mentioned protocols be encapsulated in any other of the above. For that, it's possible to transfer SNA traffic across LAN segments and over WAN/VPNs built on IP. In fact, the AS/400 must (and thus can) act as an IP and IPX router between separate network segments to be able to do this encapsulation.<br /> | Additionally with the so called ''AnyNet''-feature, OS/400 allows any of the above mentioned protocols be encapsulated in any other of the above. For that, it's possible to transfer SNA traffic across LAN segments and over WAN/VPNs built on IP. In fact, the AS/400 must (and thus can) act as an IP and IPX router between separate network segments to be able to do this encapsulation.<br /> | ||
This facility has been superseded by the ''Enterprise Extender'' feature, | This facility has been superseded by the ''Enterprise Extender'' feature, that only permits the extension of SNA networks over IP networks. Routing and Encapsulation Support for other protocol combinations was shifted to special multiprotocol routers. | ||
Newer releases also support IPv6. | Newer releases also support IPv6. | ||
| Line 51: | Line 51: | ||
== The AS/400 in a modern network == | == The AS/400 in a modern network == | ||
Today, the probably easiest solution is to connect the AS/400 machine to the ubiquitous Ethernet Network via a suitable Adapter Card. This card provides a ''line'' for the AS/400 | Today, the probably easiest solution is to connect the AS/400 machine to the ubiquitous Ethernet Network via a suitable Adapter Card. This card provides a ''line'' for the AS/400 that is to be configured with standard TCP/IP configuration parameters (Address, Netmask, Gateway). Additionally, a Domain Name System Server and Default DNS Domain can be supplied. | ||
The most important access method is ''tn5250'' to provide terminal access. This is basically a stock [[wikipedia:Telnet|Telnet]] protocol, enriched with additional functionality for providing the necessary transport features for the 5250 data stream.<ref>IBM has published [https://tools.ietf.org/html/rfc1205 RFC1205] in 1991 and [https://tools.ietf.org/html/rfc2877 RFC2877] in 2000 | The most important access method is ''tn5250'' to provide terminal access. This is basically a stock [[wikipedia:Telnet|Telnet]] protocol, enriched with additional functionality for providing the necessary transport features for the 5250 data stream.<ref>IBM has published [https://tools.ietf.org/html/rfc1205 RFC1205] in 1991 and [https://tools.ietf.org/html/rfc2877 RFC2877] in 2000 that specify technical details.</ref> | ||
The AS/400 supports the configuration of Remote Printer Queues, being implicitly served by so called ''printer writer'' applications. These pick up spooled files, usually convert them to [[wikipedia:Printer Command Language|PCL]] formatted ASCII data on the fly and transfer them to a [[wikipedia:Line Printer Daemon protocol|LPR]] compatible printer, or host.<br /> | The AS/400 supports the configuration of Remote Printer Queues, being implicitly served by so called ''printer writer'' applications. These pick up spooled files, usually convert them to [[wikipedia:Printer Command Language|PCL]] formatted ASCII data on the fly and transfer them to a [[wikipedia:Line Printer Daemon protocol|LPR]] compatible printer, or host.<br /> | ||
| Line 61: | Line 61: | ||
ODBC is also a nice way to utilize the AS/400 green screen interface basically as database file content editor, while programs or scripts on a Linux machine use this data to do some actual work.<ref>As you will see, it's relatively easy to create forms and accompanying programs for editing table content. It's definitely a lot less effort compared to achieve the same functionality in Linux with the [[wikipedia:curses (programming library)|Curses Library]].</ref> | ODBC is also a nice way to utilize the AS/400 green screen interface basically as database file content editor, while programs or scripts on a Linux machine use this data to do some actual work.<ref>As you will see, it's relatively easy to create forms and accompanying programs for editing table content. It's definitely a lot less effort compared to achieve the same functionality in Linux with the [[wikipedia:curses (programming library)|Curses Library]].</ref> | ||
It's also possible to use ''ftp'' to the AS/400 to get and put content. This is especially useful when handling ''save files'', | It's also possible to use ''ftp'' to the AS/400 to get and put content. This is especially useful when handling ''save files'', that are the AS/400 equivalent of a ''tar file'' and must reside in the ''QSYS.LIB'' file system. Also FTP can be utilized for uploading binary stream files (graphics) that could be embedded in HTML files served by the ''[[IBM HTTP-Server for AS/400 Configuration|IBM HTTP Server for AS/400]]''.<br /> | ||
The FTP server on the AS/400 also allows the execution of commands (those to be typed in a green screen command line), as a possible replacement for the rexec-facility | The FTP server on the AS/400 also allows the execution of commands (those to be typed in a green screen command line), as a possible replacement for the rexec-facility that is broken or even non-existant in most modern Linux distros. | ||
Parts of the [[Filesystems and Files on the AS/400|Filesystem]] can be exported via ''NFS'' to access data more directly. The NFS implementation on V4R5 is known to be painfully slow, though. Multiple seconds until a small directory list is displayed and transfer speeds in the low double-digit KBytes/s range are quite normal. | Parts of the [[Filesystems and Files on the AS/400|Filesystem]] can be exported via ''NFS'' to access data more directly. The NFS implementation on V4R5 is known to be painfully slow, though. Multiple seconds until a small directory list is displayed and transfer speeds in the low double-digit KBytes/s range are quite normal. | ||
| Line 69: | Line 69: | ||
=== Charset handling === | === Charset handling === | ||
An obstacle to keep in mind is the the different character set of the AS/400 vs. todays | An obstacle to keep in mind is the the different character set of the AS/400 vs. todays common systems. The AS/400 utilizes the [[wikipedia:EBDIC|EBCDIC]] character set that was invented by IBM in the punch card age, whereas common platforms utilize character sets based on [[wikipedia:ASCII|ASCII]] and it's multibyte successors. | ||
Terminal access, FTP mode ASCII transfers<ref>This comes in handy when handling large amounts of source code.</ref> and ODBC do the necessary conversions invisibly in the background. Users have no need to care. Lpr does this when ''host transform'' is enabled in the printer queue configuration. NFS does no conversation and handles everything as binary data. | Terminal access, FTP mode ASCII transfers<ref>This comes in handy when handling large amounts of source code.</ref> and ODBC do the necessary conversions invisibly in the background. Users have no need to care. Lpr does this when ''host transform'' is enabled in the printer queue configuration. NFS does no conversation and handles everything as binary data. | ||
Modern | Modern common systems utilize UTF-8 as character set. Special chars will be encoded with two or more Bytes, while as ASCII utilizes only a single Byte.<br /> | ||
Documentation for OS/400 mentions DBCS, Double-Byte Character Support, | Documentation for OS/400 mentions DBCS, Double-Byte Character Support, that is the IBM-Flavor of UTF-8. It seems that this support needs a special installation media set of OS/400. | ||
If the client for above mentioned services supports local conversion, there should be no problems for standard characters and most umlauts from Single- to Double-Byte and vice versa. This is the case with most tn5250 emulators. | If the client for above mentioned services supports local conversion, there should be no problems for standard characters and most umlauts from Single- to Double-Byte and vice versa. This is the case with most tn5250 emulators. | ||
| Line 83: | Line 83: | ||
=== A word on data encryption === | === A word on data encryption === | ||
The AS/400 actually ''can'' encrypt data. The OS supports telnet/ssl and https. Given the rapid development in cryptographic knowledge, the crypto support in older OS/400 versions is so horribly outdated and possibly crippled by former US export restrictions that it's best to stop even considering to utilize crypto on the box itself. | The AS/400 actually ''can'' encrypt data. The OS supports telnet/ssl and https. Given the rapid development in cryptographic knowledge, the crypto support in older OS/400 versions is so horribly outdated and possibly crippled by former US export restrictions that it's best to stop even considering to utilize crypto on the box itself. Even worse, crypto implementations on common systems may fail to negotiate a connection, because old, weak and known-to-be-broken algorithms offered by OS/400 aren't accepted and newer algorithms aren't supported by old OS/400 releases. | ||
If one likes the idea of exposing AS/400 services to the internet, it's best to use proxy services in front of the machine. A Linux box with a recent Distro and Apache or Nginx can | If one still likes the idea of exposing AS/400 services to the internet, it's best to use proxy services in front of the machine. A Linux box with a recent Distro and Apache or Nginx can offer SSL services to the outside world and use plain http to talk to the AS/400 in the (hopefully) secure internal LAN.<br /> | ||
Same goes for tn5250 access: Stunnel works fine in the same manner | Same goes for tn5250 access: Stunnel works fine in the same manner, telnet/ssl to the outside world, plain telnet inside. | ||
It's very likely that a near-20-year-old OS/400 may have security holes. Searching the CVE database reveals not all too much but that maybe because the AS/400 is not exactly a well-known hacker target. Please keep this in mind when deciding to expose AS/400 services to the Internet. | It's very likely that a near-20-year-old OS/400 may have security holes. Searching the CVE database reveals not all too much but that maybe because the AS/400 is not exactly a well-known hacker target. Please keep this in mind when deciding to expose AS/400 services to the Internet. | ||
Latest revision as of 22:59, 12 March 2020
Considering the heritage of the AS/400 clarifies that this system was initially designed to be the sole computer ever needed for an arbitrary business's, business administration department. The now ubiquitous (at least indirect) connection of all systems to the Internet wasn't even an idea at that time.[1]
The platform was designed primarily to support commercial business staff and decision makers. Applications provided stock management, customer master file management, accounting, dunning process management, order management, report creation for upper management, and more.
So, data was mainly typed into the system by staff and got out of the system as printed bills and reports. Support of these functions was covered in part by functions provided by OS/400 and in part by application programs from so called independent software vendors or ISV's.
Data transfers also took place with magnetic tape and diskette media carried back and forth. Magnetic tape was used as distribution media for OS/400 as well as a way to save and restore the system (backup). Diskettes have been used for transfers of remittance data to, and statements of account back from the bank.
Networking
Networking was also provided with different kinds of serial data transmission over WAN. This included SDLC and X.25, either over leased lines or switched telephone lines and modems.
The primary high-level protocol suite utilized for data transfers was Systems Network Architecture, in short SNA. These highly abstracted protocol descriptions were borrowed from the mainframe world, and enhanced to support a non-centralised communications infrastructure, independent of the existence of a mainframe computer.
LAN Attach
From the beginning on, the AS/400 provided options for LAN attach. First via Token Ring Adapters and starting in the early 1990s also via Ethernet.
The Advent of IP
Support for inter-vendor communications was provided with the licensed product TCP/IP Connectivity Utilities for AS/400, introduced by IBM in 1990.
Today, IP on the AS/400 allows a multitude of communication tasks utilizing IP instead of SNA, that is way better understood by newbies in the IBM world of networking, then SNA.
Even though that IBM moved some IP processing into the corresponding I/O processor(s), IP transfers have a comparatively high impact on CPU load. Slower machines like the 9401-150 benefit greatly from utilizing SNA for bulk data transfers, since SNA hogs about one third of the CPU compared to IP.
Terminal accesses transfer only tiny amounts of data, and these transfers take only a fraction of a second. Most of the time, terminals do nothing but show data or collect user input for later transmission. For that reason, utilizing tn5250 over TCP/IP has no noticeable impact even on a comparatively slow 9401-150.
Communication options of the AS/400
OS/400 V4 can be configured to „talk“ these protocols:
- SNA,
- IPv4,
- IPX.
It is not clear if the NetServer feature[2] also provides support for NetBIOS Frames protocol, and that releases of OS/400 support this, if any.
Additionally with the so called AnyNet-feature, OS/400 allows any of the above mentioned protocols be encapsulated in any other of the above. For that, it's possible to transfer SNA traffic across LAN segments and over WAN/VPNs built on IP. In fact, the AS/400 must (and thus can) act as an IP and IPX router between separate network segments to be able to do this encapsulation.
This facility has been superseded by the Enterprise Extender feature, that only permits the extension of SNA networks over IP networks. Routing and Encapsulation Support for other protocol combinations was shifted to special multiprotocol routers.
Newer releases also support IPv6.
Very old releases (maybe V2, V3) of OS/400 could be enabled to support AppleTalk with an official Patch from IBM. There also existed an accompanying LocalTalk adapter for SPD-slot based machines. Together with the software client from the SNA•ps program suite, Macintoshes running System 7 could access AS/400 terminal services as well as provide a printer session to the AS/400. This enabled the AS/400 to utilize the Apple LaserWriter for printed output.
Network enabled services
Numerous services were especially built or extended, so they can be utilized over the network.
SNA
Network features running over SNA include but are not limited to:
- DRDA: Distributed Relational Database Architecture,
- SNADS: Systems Network Architecture Distribution Services, similar to UUCP,
- Terminal Passthrough, very similar to Telnet.
At some point, IBM created an AS/400-Version of their own Office Application Program Suite[3], providing calendars for appointment scheduling, a word processing facility and messaging (Mail). Information could be shared between multiple systems and utilized SNADS for transferring data.
The AS/400 in a modern network
Today, the probably easiest solution is to connect the AS/400 machine to the ubiquitous Ethernet Network via a suitable Adapter Card. This card provides a line for the AS/400 that is to be configured with standard TCP/IP configuration parameters (Address, Netmask, Gateway). Additionally, a Domain Name System Server and Default DNS Domain can be supplied.
The most important access method is tn5250 to provide terminal access. This is basically a stock Telnet protocol, enriched with additional functionality for providing the necessary transport features for the 5250 data stream.[4]
The AS/400 supports the configuration of Remote Printer Queues, being implicitly served by so called printer writer applications. These pick up spooled files, usually convert them to PCL formatted ASCII data on the fly and transfer them to a LPR compatible printer, or host.
By sending data to a LPR queue on a host instead of a printer permits the usage of external tools to convert printed output to PDF format.
To utilize the database facility from remote hosts without manual labor in a forms interface, the AS/400 exposes the database files via ODBC and SQL as a facility called DB2/UDB.[5] This allows bulk-import/export of data.
ODBC is also a nice way to utilize the AS/400 green screen interface basically as database file content editor, while programs or scripts on a Linux machine use this data to do some actual work.[6]
It's also possible to use ftp to the AS/400 to get and put content. This is especially useful when handling save files, that are the AS/400 equivalent of a tar file and must reside in the QSYS.LIB file system. Also FTP can be utilized for uploading binary stream files (graphics) that could be embedded in HTML files served by the IBM HTTP Server for AS/400.
The FTP server on the AS/400 also allows the execution of commands (those to be typed in a green screen command line), as a possible replacement for the rexec-facility that is broken or even non-existant in most modern Linux distros.
Parts of the Filesystem can be exported via NFS to access data more directly. The NFS implementation on V4R5 is known to be painfully slow, though. Multiple seconds until a small directory list is displayed and transfer speeds in the low double-digit KBytes/s range are quite normal.
The TCP/IP support for OS/400 includes a lot more services but these are rarely needed and used.
Charset handling
An obstacle to keep in mind is the the different character set of the AS/400 vs. todays common systems. The AS/400 utilizes the EBCDIC character set that was invented by IBM in the punch card age, whereas common platforms utilize character sets based on ASCII and it's multibyte successors.
Terminal access, FTP mode ASCII transfers[7] and ODBC do the necessary conversions invisibly in the background. Users have no need to care. Lpr does this when host transform is enabled in the printer queue configuration. NFS does no conversation and handles everything as binary data.
Modern common systems utilize UTF-8 as character set. Special chars will be encoded with two or more Bytes, while as ASCII utilizes only a single Byte.
Documentation for OS/400 mentions DBCS, Double-Byte Character Support, that is the IBM-Flavor of UTF-8. It seems that this support needs a special installation media set of OS/400.
If the client for above mentioned services supports local conversion, there should be no problems for standard characters and most umlauts from Single- to Double-Byte and vice versa. This is the case with most tn5250 emulators.
If the AS/400 in a Single-Byte Character-Set Installation is supposed to do the conversion, one cannot utilize UTF-8 on the client side but must make sure to handle data with the Latin1 (ISO-8859-1) Code Page. This is the case with FTP/ASCII transfers, and LPR with host transform.
It's not known to me who's handling the conversion when accessing data via ODBC.
A word on data encryption
The AS/400 actually can encrypt data. The OS supports telnet/ssl and https. Given the rapid development in cryptographic knowledge, the crypto support in older OS/400 versions is so horribly outdated and possibly crippled by former US export restrictions that it's best to stop even considering to utilize crypto on the box itself. Even worse, crypto implementations on common systems may fail to negotiate a connection, because old, weak and known-to-be-broken algorithms offered by OS/400 aren't accepted and newer algorithms aren't supported by old OS/400 releases.
If one still likes the idea of exposing AS/400 services to the internet, it's best to use proxy services in front of the machine. A Linux box with a recent Distro and Apache or Nginx can offer SSL services to the outside world and use plain http to talk to the AS/400 in the (hopefully) secure internal LAN.
Same goes for tn5250 access: Stunnel works fine in the same manner, telnet/ssl to the outside world, plain telnet inside.
It's very likely that a near-20-year-old OS/400 may have security holes. Searching the CVE database reveals not all too much but that maybe because the AS/400 is not exactly a well-known hacker target. Please keep this in mind when deciding to expose AS/400 services to the Internet.
Footnotes
- ↑ The predecessor of today's Internet — the ARPANET, switched to today's known TCP/IP protcol suite starting in 1983.
- ↑ Basically a piece of software that provides NetBIOS-Services: File Sharing and Printing. Kind of Samba from IBM.
- ↑ DISOSS for MVS and VSE, PROFS for VM, Office/36 for the System/36.
- ↑ IBM has published RFC1205 in 1991 and RFC2877 in 2000 that specify technical details.
- ↑ By now you may perhaps be aware that the database is an integrated and unseparable facility of the AS/400. The DB2-Part is mostly some kind of translator between SQL and calls to the operating system for reading and changing of database files.
- ↑ As you will see, it's relatively easy to create forms and accompanying programs for editing table content. It's definitely a lot less effort compared to achieve the same functionality in Linux with the Curses Library.
- ↑ This comes in handy when handling large amounts of source code.