My father worked on a product called SnapVantage in the late 90s and early 2000s, which basically used the copy-on-write abilities of the virtualized mainframe disk in a StorageTek disk array combined with z/Linux and z/VM as a hypervisor to instantly provision and boot Linux machines.
It looked like a very, very toned down version of the AWS management console, with a near-identical ability to select a distribution, upload SSH keys, find an IP, and have a full, running VM in seconds... but this was in 1999, when watching a Linux box "copy," provision, and come up in just a few seconds was mind-blowing.
I used my nascent Perl skills (as a ten-year-old) to write the installer for the host software, which I believe actually shipped to customers nearly unmodified!
The zSeries still supports a lot of battle-hardened HA features like lock-step execution, hot CPU fail-over, and hot-swap memory and IO which are very rare, esoteric, and hence hard to support in the x86 world (and rare, esoteric, and untested is not what you want in your HA system).
Plus, the number of VMs that can run well on a z/Linux and z/VM environment is amazing - we had hundreds and hundreds of instances running on one zSeries.
There's a very full-featured zSeries emulator called Hercules which will allow you to run z/OS and z/VM (if you can find them, of course). There's also a distribution of MVS 3.8j, the 1980s System 370 / System 390 precursor to z/OS, available for Hercules for free if you want to get a taste of what administering such a system is like. A lot of obscure job-scripting languages like JCL and early scripting languages like REXX were developed for early mainframes and still live on today.
My understanding is that these machines are amazing at IO, and handle virtualized environments incredibly well (and have so for decades) (e.g. thousands of Linux machines running).
If you need raw CPU performance then a mainframe is not for you. If you need competitive pricing then a mainframe is not for you. If you have a mainframe and need a have an expiring agreement then this machine may be for you. If you have legacy mainframe code then this machine may be for you.
I remember watching a financial update from Sam Palmisano (IBM's previous CEO). I swear he started to drool when referring to IBM's mainframe margins. From my experience mainframes are for legacy companies with legacy systems, as they are not price competitive with other markets that experience real market competition. But, it is interesting and incredibly reliable hardware.
What makes them amazing at IO?
Isn't IO more a result of what type of disks a computer has? An SAS 15k disk can only do 300 iops whether it's a mainframe or a commodity server, no?
Lots of cache memory and very fast buses to very large external disk arrays. An old mainframe trick was to use the previous generation hardware as the IO controller for the current generation...
And this is a theme replicated throughout mainframe design. Everything is replicated, all pathways are wide as possible, everything is integrated from the hardware up to the application layer.
Take virtualisation, for instance. IBM mainframes have PR/SM, a hardware-level capability to virtualise everything in the system down to the lowest level of granularity. And they've done this since 1985.
A SATA port can do some number of IOs per second (this depends on how the controller is designed for the most part) and the SATA data pipe is 6 gigabits/second or 600 megabytes per second these days.
A SATA Disk (the spinning rust kind) is different, it can pump out some maximum amount of data per second (usually about 100 Mbytes/sec and has an average seek time of a few milleseconds. So if you are doing random i/os to the disk, so that every I/O needs a seek, and the seek takes 10 mS, you can do at most 100 I/O operations per second (IOPS) if you transfer 512 bytes with each IOP you get a throughput of 100* .5K or 50K bytes per second. If you transfer 100Kbytes per transaction you get 10 megabytes per second. Anyway the bottom line is that an I/O channel is bound both by its bandwidth and the number of IOPS it can do.
A PCIe 'lane' can do 250Mbytes per second and with Intel's 5500 chipset about 2 million IOPs. (Although those IOPs are spread across all PCIe channels) A "big" server chip set will give you 32 or maybe 48 PCIe "lanes" which can be 12 by 4 (or x4) lanes or 2x16 + 8x1, or 2x16, 1x4 and 4x1 (this is a common desktop config for two video cards and misc peripherals)
The bottle neck is exactly like networking, each I/O device can consume a certain amount of channel bandwidth, since they don't run 24/7 you can "oversubscribe" which is to say potentially have more I/O devices than you have bandwidth to service them (this happens a lot in like big disk arrays) and ultimately the transaction rate of the entire system is constrained by the cross sectional bandwidth of memory and I/O channels such that you get I/O limited in terms of things you can push around.
Anyway, a mainframe has thousands of channels. Lots and lots of them. And each of those channels has a lot of raw bandwidth and can sustain a high transaction rate. This makes a mainframe a transaction monster.
In the taxonomy of computer systems, 'supercomputers' have a monster amount of memory bandwidth and high transaction rates (low latency) between their compute nodes (or cores). Mainframes have a monster amount of I/O bandwidth with high transaction rates between their nodes and I/O devices. General purpose computers achieve good economics by limiting memory and I/O bandwidth to stuff that will fit in a single piece of silicon. (or at most a couple of pieces)
Of course, mainframes are also faster simply because they are more expensive (e.g. 100 PCIe slots are obviously faster than 10 slots), but that's not an intrinsic advantage (in theory you could develop a $10M x86 server — call SGI if you're interested).
That said, they do deal with IO pretty well. They have dedicated offload processors for transferring data, which means for the same (IO intensive) workload, the CP utilisation of a zSeries machine would be much less than that of an x86, pSeries (or other) machine. And in terms of the disk being a bottle neck... If you're pushing to a disk array that has 192GB (pretty standard) of write-back cache... then no, the disks are not a large concern.
But... At the end of the day, does this really weigh up to the cost of Big Iron, or for that matter the additional licensing of software on top of it?... Not really.. is my answer.
I don't actually think performance wise they have anything on a cluster of x86 servers. But its hard to benchmark things like that accurately.
From a developer point of view, its not a great technology stack to work on. Its old and out dated, and is pretty incompatible with other platforms. EBCDIC is a pain, 3270 terminal is a pain. Having to allocate files manually is a pain. Having actually about 7 different types of files is a pain.
Its really all a pain. Just dont go there.
Also, I found that guys who had been working on mainframes for a really long time have a pretty outdated view of what the "distributed" world has to offer. A lot of the time they dont understand what "web Scale" at the level of google, facebook et al are operating at.
As an additional funny annecdote, the I/O that a lot of people have been talking about is now conventional if high end disk array's AFAIK. But internally the disks are emulated for the OS to look like the disks it was using back in like 1975. Thats how it gets backwards compatibility, just dont change anything.
Can you imagine a time 1000 years into the future where software written today might still be in use? How cool would that be?!
It's a bit like the descendants thing that was discussed a few days ago on here - either lots of stuff will survive the next 1000 years (and therefore lots of running code will) or almost nothing will.
To name the ones I know of - vi, emacs, C, Common Lisp. These have held pretty steady for about 30 years now.
If banks still exist they'll still be emulating zOS on their quantum dimension-folded computers...
You hit it. Things break when you change things. The high priesthood that surrounds all mainframes prevents that from happening.
Nowadays, if you have to change stuff, IBM sells you AIX boxes that cost nearly as much as mainframes which sort of integrate with them to support those changes.
Basically it's a different universe. Everything that might be familiar to someone with a unix or windows background is ... different. More eldritch. More powerful and yet more obtuse.
Oh and JCL. JCL is the devil's work.
One thing they did that I thought was cool was to consolidate a bunch of Linux servers onto their mainframes. They replaced a few racks with z/VM instances. If they needed another server, they could spin it up in literally seconds if the template was already loaded. This was back in 2004, 2005.
You don't buy (actually: rent) a mainframe (actually: CPU cycles and RAM capacity) because it's cost-effective. You do so because the system absolutely must work and you have a mind-numbing number of transactions to process in a hurry.
I can't recall any specific stats or stories right now but you can imagine the discussions between a father who develops primarily with Assembly and a son (me) who ended up being a .NET developer working on various startups. We've had a few talks about the 'cloud' mostly resulting in him laughing at what we call 'web scale'.
TPF is an OS, transaction processing & database system that was written in mainframe assembler and deployed in 1979 to run SABRE and a few other core transaction systems (VISA, AMEX, Marriott hotels, etc.) - maybe 100 customers still use it today - and it was a descendent of a previous assembler system developed in the 1960's.
It probably was (and is) the closest system to "web scale" through the 80's and 90s's in that it ran some of the highest volume workloads globally (e.g. 5000-10000+ tps back in the 80s / 90s, 25000+ tps today) at 99.99% availability. In context, Twitter hits 10-15k tweets per second during big sports events.
These days it's extremely expensive and slow to make changes to TPF relative to modern technology (as it traditionally required S/370 or S/390 assembler, though these days requires C). IBM has a newer version called z/TPF which makes it easier to code in C or C++ and communicate with the outside world via web services, etc.
A very different world...
maybe 100 customers still use it today
Indeed it does seem like there are only a few companies that use this stuff. Like you said, you had to be a pretty big company with massive amounts of transactions to merit it. He's actually contracting with the IRS right working on some newer systems to handle your tax returns. (He tells me every year it's a miracle you get your tax returns every year on the old system)
Whenever he's looking for a new gig he basically has a set list of contacts he goes through. It seems like almost all the developers of these systems know each other.
I remember being at a startup and being impressed that we were doing a million dynamic pages per hour. Then I got the chance to see a real system at a big company pumping out orders of magnitude more traffic than that per second. (I am not at liberty to share exact numbers.) And doing constant processing on that traffic.
I doubt your father would laugh at systems like that if he learned what they manage to handle.
Let's just say that any semi-reasonable back of the envelope will give you some truly impressive numbers. I don't think that I should say any more than that.
Now consider that this data is constantly being processed to figure out things like click through rates, trending requests, and much, much more. And that data is then fed back into the live site.
The implied transaction rate should be enough to impress the average mainframe user.
[1] https://en.wikipedia.org/wiki/Mainframe_computer#CharacteristicsThis one is worth listening to, for learning a bit about mainframes: http://www.se-radio.net/2012/03/episode-184-the-mainframe-wi...
In any case, I'd advise anyone who can have a try at mainframe to give a look, even if just because it's one of the still used non-Unix based systems.
Most companies are trying to stop using the mainframe so much, it is laughable though because they are moving from mainframe based solutions, to mainframe based solutions.