Saturday, January 10, 2015

How to solve simple linear equations

It may come as a surprise to some people, but in high school this guy was not good at math at all. The point is that while I'm pretty good in designing algorithms, my calculations are always a bit off. Nowadays that's not much of a problem: you just grab a calculator or (even better) a computer, but the school system in those days was focused on doing it by hand. It took ages to calculate and plot a graph manually, which in my case was already completely clouded by tiny calculation errors. On top of that, it was too time consuming to make only one graph, let alone a whole series of them, so I couldn't get a feel of the dynamics involved.

Fortunately, that all changed when the computer came in. And then I started to understand all those things that had eluded me for years. Like solving linear equations.

There are several methods to solve them. One involves eliminating terms by applying the same operation at both sides of the equation. When I was a youngster, that only resulted in more complex equations than the ones I started with. Sometimes it didn't even resemble the original equation. The other one is switching terms from one side of the equation to the other side, while changing the sign. That one is already much better and the foundation of the method I will be presenting here.

You only need to fill in a simple table and then do some additions. If you manage to do that, you're left with one single division - and that's it. Let's take the following equation:
2x + 2 - 76 = -12x + 10
The table is a no-brainer. Make two columns and name the first one "+x" and the other one "-n". Got that? Note that even if you don't get the signs right, it's not a problem. Now, fill in the numbers associated with "x" in the left column and all others in the right column. If a number is on the right part of a subtraction, add a minus sign, e.g. "12 - 4x", will result in "12" in the right column and "-4" in the left column.

Once you hit the "equals" sign, subtotal both columns. Now write two new labels, "-x" in the left column and "+n" in the right column. Then continue until you're done. So now we've got something like this:

Now negate the number in the top right column and the bottom left column and add the subtotals in both columns:
(2 + 12)x = 74 + 10 =>
14x = 84
I promised you, you would be left with one single division. I didn't lie, here it is:
x = 84/14 =>
x = 6
That's it! It's so simple even a computer can do it. And that's exactly what I did in just 40 lines of Forth code - and that includes the entire parser:

Sure, you always risk making some errors while adding or dividing, but apart from that it's pretty bulletproof. So if your kid is eluded by linear equations, give it a try! Just make sure he (or she) understands that equal signs ("+ +" or "- -") make up a positive number, while unequal signs ("+ -" or "- +") result in a negative number.

But I'm sure he (or she) will manage that - since I mastered that one too forty years ago! ;-)

Sunday, October 20, 2013

Hardening your Raspberry Pi

When I had traded in my Microsoft box for a Linux box, I wanted it to look and behave like the large $100K Unix boxes I had had the pleasure to work with before. For instance, I bought a tape unit to make backups. It was terribly slow, but I kind of liked the whirring of the tape when you entered rewind at the console. That was cool.

So I was overjoyed when my brother came up with an old Wyse25 serial terminal. It took some fiddling with serial cables, inittab and settings, but somehow I got the beast to cooperate with my Linux box. It silently displayed kernel messages for years.

Sometimes, when my box hang, I logged in to shut it down properly. And at one time it started to spit out "Sense errors" from my SCSI drive. That got my attention. I backed it up and restarted - but it never came up again. Fortunately, my data was safe.

But after a decade, the Wyse25 began to behave erratically. The screen blinked and became unreadable at times. Most of the time a good whack fixed it - until the moment that was no longer sufficient. Maybe it was the dust that had accumulated over time, I dunno. Anyway, I needed a replacement.

But nowadays these things are hard to get. They are out of production and even a refurbished one would set me back a few hundred quid - and I needed it shipped overseas. That wasn't going to be cheap because these things are heavy, darn!

After a while I decided a Raspberry Pi and a cheap monitor would do the job just fine, setting me back about 200 Euros. I already had several parts lying around, like a keyboard and a mouse. It would not only allow me to show kernel messages, I could use it as a cheap workstation as well. That sounded like a good deal, so I ordered one.

It came in after a few days. Setting it up was a breeze (although I could not get it to accept the full 1080p resolution - but who cares) and within 20 minutes the kernel messages were scrolling down the screen. I pimped my 16G SD image with LibreOffice and a few other packages that seemed useful at the time and was quite happy with my low-cost workstation.

I put the monitor in "ECO-mode", so the whole contraption was using a mere 18W (which was 10W less than my Wyse25). Of course the Pi was on 24/7, because I wanted it to be available straight away - booting it every time would defeat the whole purpose. Problem solved. I thought.

After a few months I wanted to demonstrate the thing to a friend of mine, so I started up LibreOffice. Nothing. I rebooted it. It came up, but with errors. Fortunately ssh was still working, so it was not too big a problem. Until another reboot several months later when it just showed the dreaded kbd> prompt. That was no good..

I started googling and found out I had become the victim of "SD card corruption". It seemed to be quite common. Fortunately, there are two partitions on a Raspi SD card and my config.txt was on the first VFAT partition, so I could refrain from setting up my monitor all over again. This time I used another, smaller class 10 card and concentrated on its primary purpose - which was a console. Within half an hour I was up and running again.

But where did this corruption come from? I had been using SD cards for years (e.g. on my EeePC 701) and had never experienced corruption. Good powersupply? Got it! Overclocking? Not me. Good quality SD card? Sure! Power failure? Not that I can remember. Pulling the powercord? Are you nuts - I was an system admin! Still, I didn't want to repeat that procedure over and over again. So what could I do?

There are several things I don't understand about the Raspbian distribution:

  • Why is there no separate /home partition;
  • Why is there no separate swap partition;
  • Why use a journaling filesystem, EXT4.
With a bit of tinkering, one could even mount the root filesystem read-only and temporarily remount it read-write if one installed new packages. But - that was not the case. I had to work something out myself.

Note that these instructions are for the 2013-09-25 Raspbian distribution, which has already been tweaked to minimize SD card corruption by setting noatime to the root filesystem and moving a whole lot of stuff to tmpfs. But more could be done.

The first thing was to remove the swapfile. A box that just does a little ssh would probably not need it. That was simple.

sudo dphys-swapfile swapoff
sudo dphys-swapfile uninstall
sudo update-rc.d dphys-swapfile remove

The next thing was to move /tmp to memory. Just issue:

sudo vi /etc/default/tmpfs

And then set this parameter:


Finally, I wanted the logfiles in memory too. Note that they quietly disappear every time you shutdown, so they are of very little use when your Raspi has crashed. Just issue:

sudo vi /etc/fstab

And make sure it looks like this:

proc /proc proc defaults 0 0
/dev/mmcblk0p1 /boot vfat defaults,noatime 0 2
/dev/mmcblk0p2 / ext4 defaults,noatime 0 1
logfs /var/log tmpfs size=10M,noatime 0 0

However, this is not enough to get rid of all writes to the SD card. resolv.conf will still be rewritten each and every time - and of course, we still got the journal. But - it's not too bad for a start.

The Raspi has been working without a hitch for several weeks now. Rebooting was not a problem. But I still feel the whole story will pop up again some time or another, so I have been contemplating what my next move will be. The most painless one is to do what I suggested before: make the root filesystem read-only. There is even a special Raspbian for that one.

You could also try using a USB stick or USB harddisk as well. Just be sure that the Raspi gets enough power, which often means you need a powered USB hub. Another drawback is that you have to fork out a load of cash for this solution and that I've read reports about corrupted USB sticks as well. No guarantees here, so I can't really recommend that.

The bottom line is that the Raspi can boot from a whole bunch of devices, so you're free to make your own variant of Raspbian (e.g. on EXT2) or even try booting from NFS. It's all not too difficult as I will show you in the next post, where I'll explain how to move Raspbian to another medium - and how to emulate it. Which I do as well, because I don't feel happy developing on a physical Raspi for obvious reasons.

Although some may claim I have given you the impression that the Raspi is unreliable or a money pit, be assured that is clearly not my intention. I think it's a very nice machine with an incredible value-for-money, but it may need some tinkering to make it work for you in a particular situation, especially if you use it for production purposes - like a server that has to run 24/7.

And that's the beauty of the whole concept: you can!

Friday, May 4, 2012

My life with Coherent, part 2

In the first post of these series I've shown you it's possible to make Coherent 4.2.10 run under the newest version of QEMU. Now it's time to take it one step further.

I've had permission from Robert Swartz himself to distribute the first perfectly legal Coherent 4.2.10 QEMU image as long as you comply with the following conditions:
You may use the software on this image free of charge for personal, non-commercial use. You may NOT redistribute this image or the software it contains without written permission from the copyright holder. The software is provided on an "as is" basis without warranty of any kind.

Getting started
Download and unpack the archive. Included are two images, coherent.public.img and fat16.dd. The first one contains the Coherent image, the second one an empty floppy. If you want to, you can start them right away:
qemu-system-i386 -hda coherent.public.img -fda fat16.dd -m 16
First, don't try to crank up the amount of memory, Coherent won't use it - at least not this kernel. Second, you just have to be patient to attempt this, because it may take up to 10 minutes before Coherent is up and running. The good news is that you can monitor how far it is by pressing [CTRL]-[ALT]-2 in QEMU and typing:
info blockstats
If rd_bytes reaches 600,000 you can probably login. There are two login entries, the obvious root with password rootroot and a normal user named habe with password habehabe. Pick any one.

When you've logged in you might want to try the usual stuff like ls -l or ps -eaf, maybe you're even so bold to start up vi or cc. In any case, you will find that everything is painfully slow. If you want to get out, you can do so, but you'll have to be root in order to do it properly. Just issue:
cd; /etc/shutdown halt 0
Note it doesn't shutdown immediately! You'll have to wait until Coherent tells you it's safe to power down. Be patient! You can also exit QEMU without shutting down, but in that case you may have corrupted your image. And believe me, at these speeds you do not want to run fsck..

So that's it? Coherent runs, but it is completely unusable? No, not quite. Coherent does some disk caching, so after a while it becomes responsive. That is: until you reboot. But how can you avoid a reboot? Well, it's quite simple. QEMU allows you to make snapshots, so if you need Coherent you don't boot it, but simply load the snapshot.

QEMU only allows snapshots when you're using the so called "qcow2" image type, so we have to convert the files you just downloaded. That's not too hard, since QEMU comes with a handy utility for that. Just issue:
qemu-img convert -c -O qcow2 coherent.public.img Coherent.dsk
qemu-img convert -c -O qcow2 fat16.dd fat16.dsk
The latter may give you a warning, but don't worry, we'll only use it as a placeholder. Ok, we're done, let's start it up again:
qemu-system-i386 -hda Coherent.dsk -fda fat16.dsk -m 16
Agreed, you're in for the same wait. Sorry for that. But I promise you next time it'll be much faster. Just stay with me for a little while longer. When Coherent comes up, log in as usual. Then press [CTRL]-[ALT]-2 to enter the monitor. Now type:
savevm test0
This will save the current state of Coherent in a QEMU snapshot, so you'll end up here next time you start it. Return to the emulation by pressing [CTRL]-[ALT]-1 fiddle around a little bit, start up your favorite commands a few times and then save a new snapshot:
savevm test1
Shutdown Coherent properly and start it again, but this time with the command:
qemu-system-i386 -hda Coherent.dsk -fda fat16.dsk -m 16 -loadvm test1
Ok, that's more like it, isn't it? Ok, shut it down again. This time you'll be able to start it up much faster. Let's try something fancy now and invoke QEMU with:
qemu-system-i386 -hda Coherent.dsk -fda fat16.dsk -m 16 -serial telnet:localhost:4444,server,nowait -loadvm test1
Log in as root and type:
enable /dev/com1l
Now start up a terminal session on your host and type:
telnet localhost 4444
Yes, you can login to Coherent from your host Operating System. But that's about everything you can do as far as networking under Coherent is concerned.

There is a full TCP/IP stack available for Coherent, but you'll need a special kernel in order to make it work - which doesn't seem to be available. So I haven't succeeded to do much more in this area. If you have, please leave a comment.

Installing Coherent
Maybe you'll think this small system is so much fun that you want to give it a permanent place on your desktop. That is where the script comes in. It works with both KDE (KDialog) and Gnome (Zenity) and it makes it easier to launch the QEMU/Coherent combination. The only thing you have to do is to state where your disks are located:
If you want to exchange files between Coherent and your host, you'll have to change /etc/fstab as well:
/opt/qemu/fat16.dd /mnt/coherent msdos rw,user,noauto,loop 0 0
Note this is the uncompressed, original file! You can mount the uncompressed Coherent image as well (as sysv), but you won't be able to write to it. I don't think I'll have to tell you how to mount a floppy image, do I? Once the VM is up and running you can exchange the virtual, empty floppy for the raw version by switching to the QEMU monitor and issuing:
change floppy0 fat16.dd
However, while the raw floppy is online you can't make a snapshot. When you want to take a snapshot you simply return to the dummy floppy by issuing:
change floppy0 fat16.dsk
Ok, now we've cleared that one we can concentrate on the real issue here: how do we get any files from the floppy? That's not as difficult as you think. You'll only need to edit one file, /etc/default/msdos. Make sure the appropriate section reads:
# This is for a system with a 3.5 inch A drive and a 5.25 inch B drive
And you're ready to rock 'n roll. There is a whole slew of MS-DOS related commands at your disposal like:
  • dosls
  • doscp
  • dosrm
  • dosrmdir
E.g. copying the /etc/default/msdos file to the floppy is simply:
doscp /etc/default/msdos a:
And copying it back is done by:
doscp a:/msdos /etc/default
Well, that ain't rocket science, is it? We can use it right away to fix some Y2K issues. Yes, this version of Coherent won't go past the 31st of December 1999, but it is easily fixed by ATclock and date. Copy them to the /bin directory and reboot. That'll fix it.

You may also be tempted to try another kernel. Several ones are available here. Just copy it to the root and hard-link it to autoboot, e.g.:
ln cohat0 autoboot
Note you can also select a kernel when starting up - just press [SPACE] immediately. Coherent will answer:
If installing COHERENT, please type "begin".
But it is actually a prompt for the kernel name. Obviously, if you type autoboot it will boot the default kernel - just in case you don't know how to get out of there ;-)

Final notes
  • Although Coherent comes with a full fledged C compiler, it won't compile your ANSI-C sources, since it is strictly K&R C. However, GCC - and all the GNU stuff that comes along with it - is available. Still, I like the little beast, since it makes small and fast compilants.
  • Coherent does have virtual consoles. Press [CTRL]-[NUM0], [CTRL]-[NUM1], etc.
  • The archives of comp.os.coherent are compelling reading, featuring Linus Torvalds on who's got the best OS.
  • Additional notes and links on the subject are appreciated. Just leave them in the comments.

Useful links
Good sources for additional software are:
Additional information can be obtained:

There have been consistent reports on Coherent being considerably faster on QEMU 1.7 and up. Well, I tested it and even if there may be some truth to it, it's not very significant nor consistent. If you can deal with it, convert your QCOW2 image to a raw one, because that will give you the required speedup.

But the biggest drawback is that your C compiler will go haywire, spitting out errors on perfectly fine include files. This behavior was still present on the QEMU 2.0.2 version, although slightly less prominent.

The last QEMU version known to work flawlessly is 1.4.

Coherent is Open Source now (thank you, Jason Stevens)!