Slicing Pi’s differently

How it all currently is

The way my Pi’s were all setup was :-

  • markpi – Raspberry Pi OS Buster 64 bit kernel and 32 bit userland. On it I had this web server, SAMBA and was our main video player. This is a 4GB Pi 4 with a 1TB Seagate SSD as the boot and storage device.
  • raspberrypi – Raspberry Pi OS Bullseye 64 bit. It is a 1GB Pi 4. As well as about 10TB of spinning rust attached via a Simplecom 3 x 2 bay powered SATA boxes with their own power it also had a 1 TB Toshiba SSD. The Simplecom units are daisy chained using their USB ports and the SSD is in the other USB3 port of the Pi. Bandwidth and speed are not important as this is storage for my digital and scanned photos as well as the family digitised slides and Super 8 movies. About 8 or 9 TB of the stuff. All of its storage is shared via SAMBA and / or NFS.
  • Desktop – Raspberry Pi OS Bullseye 64 bit.  It is my daily driver desktop machine and runs from a Sandisk 500GB SSD.
  • mincepi – Raspberry Pi OS 64 bit Pi 3B+. Acting as my local caching only DNS via “dnsmasq” along with TVHeadend.
  • pi-zero – Raspberry Pi OS Bullseye 32bit. SSD card only and is my GPS disciplined time source with NTP. It also has a small display showing date and time only. That’s it….
  • Pi 400 – this one is sitting on the shelf and does nothing. Good to have a spare though. If any of the other Pi’s go belly-up a simple change of the boot media and plug it in and we’re away again.

So why change it ?

The biggest shortcoming was that the web server and our media player is on the same Pi. For me that wasn’t acceptable as there were conflicting demands. When playing streamed media the network traffic was intense which compromised the web server so I thought something needed to change.

The Pi 3B+ was basically wasted as all it did was dnsmasq and TVHeadend.

The Pi-zero was, and remains just a GPS disciplined ntp server.

The NAS Pi remains so although I wanted to change it to booting from spinning rust. Stuff had to moved around a bit too – there are drives that are nearly full and others that are nearly empty. For example the drive that has the old Super 8 movies on it is only about half full and the storage requirements here are not going to increase. Moving it to a smaller drive that’s only just big enough makes sense. I’ve thought about RAID but I feel that’s not the best solution here – too many things can go wrong. Better to organise the current storage so that there is free space where needed and bugger all free space where it isn’t.

At the end I want a Pi serving up this web site and TVHeadend so we can watch free to air TV in places where terrestrial TV is unavailable but mobile phone services is. I wanted a robust desktop Pi with enough storage available. I wanted my GPS disciplined clock available via NTP and I wanted a NAS with its storage organised sensibly. Lastly I wanted a Pi to play our movies and stuff without interfering with anything else. This Pi stays with Buster as Bullseye, 32 and 64bit, plays really, really badly with our TV which is connected to the Pi (I strongly the Bullseye “kms” driver) which is what we watch “stuff” on.

Surely that’s not too much to ask. Is it ?

 

The changes

The Pi 4 4GB media player has the media drive from the NAS Pi mounted in a suitable spot so basically all it’ll need is a SD card with VLC, etc. on it. It also is running Buster as previously noted. As they say in the Meerkat ad’s, Simples.

The NAS Pi, with a lot of messing around, has had the storage moved around and generally reorganised. The irreplaceable stuff is now on a single 8 TB hard drive and is backed up twice with one copy kept here and the other with a friend for safe keeping. This is essentially static storage and only added to when the family discovers new stuff – family and school photos, holiday movies, etc. The replaceable stuff, as well as our digital and scanned photos, is spread over two more 8 TB hard drives and the boot device is now a 3 TB hard drive which also holds the NFS exports for “/home” on my desktop Pi and the media player Pi. All of the re-organising left me with two 8TB hard drives spare.

The desktop Pi has the “/home” directory moved to a NFS drive from the NAS Pi and has the “/boot” and “/root” on a smaller SD card.

The Pi 3B+ is now the web, DNS (via dnsmasq) and TVHeadend server running Bullseye 64 bit with a SD card boot device.

The Pi-zero remains as my time server just for fun. It also has dnsmasq as a backup caching only name server.

All except the Pi-zero and the desktop Pi are powered with Raspberry POE+ hats which work really well and declutter the cupboard they are in considerably.

So, how did it all work out ?

Well, so far it’s all worked out pretty well. I’ve even managed to migrate my wife to a Pi. Because of my new structure it was really easy – just clone my desktop SD card and make a couple of changes – dhcpcd.conf and fstab and then plug it in and boot it. My wife didn’t need a lot of convincing either – the annual M$ licence fee impost and changes to the pricing structure which would make MS Office a LOT more expensive did that. Job done and a home found for the “spare” Pi-400.

 

So what’s happening on the Pi front

Some ssh strangeness enters the scene

We had a NBN (National Broadband Network) outage here caused by a broken cable in the copper between the neighbourhood node and our place. NBN Co got onto it pretty quickly and it was repaired within the day.

When the tech. arrived the first thing he told me was that our modem/router was no longer supported and that we needed to use the newer one that I had. The reason that we were using the old the old modem/router is that the newer model couldn’t do port forwarding or dynamic DNS. It transpired that the new model had had a firmware update and both dynamic DNS and port forwarding were now supported. This was proven to be the case and the configuration was entered into the new modem/router.

Enter the SSH problem. My two Pi4’s were connected via a hub and ethernet cable, my Pi-Zero was connected via 2.4GHz wifi and my Pi-400 is connected via 5GHz wi-fi. With the new modem I can’t SSH across connection modes. For example, I can’t SSH into any other system on 5GHz from a system on 2.4GHz or ethernet. I can, however, connect to any other system on 5GHz wi-fi. I spent days playing with this until I found a technical document which described the problem as I was seeing it. It only applies to SSH so I just use telnet now. Strangely, I can SSH into any system via jandmf.com:xxxx where xxxx is the port that’s forwarded to the desired systems port 22.

An advantage of the new modem/router is that if the NBN decides not to play nice is uses the 4G network as a backup. In the backup mode there is a huge disadvantage though. Dynamic DNS doesn’t work and neither does port forwarding.  Oh well, I can live with that as the NBN goes down very rarely.

Some PoE+ hats

Up behind the TV I had a rats nest of cables and power strips. The design of the Raspberry Pi power supply with its “wings” means that I need a few power strips rather than just one. As well as three Pi’s we have the TV and phone and laptop chargers so if I could get rid of the Pi power supplies I’d only need one power strip. I was already using an eight port PoE (Power over Ethernet) hub so all I needed was a few PoE hats which were duly ordered as were a few plastic / acrylic PoE compatible cases.

In my usual style I went for it, putting the Pi’s and the PoE hats in the new cases. Looks OK too. here’s a link to the PoE HAT and here’s the link for the PoE HAT compatible case.Straight off the bat it all worked perfectly apart from the fan going at full speed and being annoyingly loud. Clearly something needed to be done about that. Anyway I installed the PoE HATs’ and cases on the other two Pi’s  – a Pi3B+ and the other Pi4B 1GB so I now have to Pi4’s (1GB and 4GB) and a Pi3B+ with PoE working perfectly.

Next came the taming of the fans. I very nearly found the end of the internet finding what I needed. It transpired that a few lines in /boot/config.txt would do the trick.

dtoverlay=rpi-poe-plus
dtparam=poe_fan_temp0=10000,poe_fan_temp0_hyst=1000
dtparam=poe_fan_temp1=55000,poe_fan_temp1_hyst=5000
dtparam=poe_fan_temp2=60000,poe_fan_temp2_hyst=5000
dtparam=poe_fan_temp3=65000,poe_fan_temp3_hyst=5000

I assume that there are four fan speeds zero, quarter, half and full speed. I gleaned this by watching the output from “cat /sys/devices/platform/rpi-poe-fan@0/hwmon/hwmon1/pwm1”. This is the PWM value which seems to be either 0, 64, 128 or 255. I don’t know if there is any more granularity than that though.

In /boot/overlays/README there is a concise explanation of what dtparam=poe_fan_tempn lines do.

OK so that’s three Pi’s powered by PoE hats. The fans are under control and quiet and seem to run at the PWM of 64 or 128 levels with a temp of 55 to 65 degrees C which is OK as the CPU doesn’t get throttled back until it reaches 85 degrees C. Having the three Pi’s powered by PoE enabled me to put two of them in the cabinet under the TV out of sight. Bonus.

 

A Pi3B+ enters the fray

While the PoE stuff coupled with the SSH stuff was going on I thought that I’d reorganise what was where.

The Pi-400 remains as my desktop system although that’s in the process of being upgraded from Buster (Debian 10) to Bullseye (Debian 11) and so far the 64bit Beta is working quite well apart from a couple of known issues concerning Bluetooth and wi-fi.

My 1GB Pi4 is now purely a NAS server with about 10GB of attached storage on spinning rust. It is running Bullseye 64bit headless with no desktop installed. Apart from PiOS Lite I have only installed telnetd, telnet, ftpd, ftp, Samba and NFS server.

My Pi4 4GB is now a media centre and database server. The database is for this web server and is MariaDB. It’s used to play music, TV shows, etc with VLC. The media files are all on the NAS server Pi. It’s still running on Buster with 64bit Kernel and 32bit userland though. I’m waiting for Bullseye 64bit to come out of Beta before I upgrade it.

The Pi3B+ is now the web server as well as a TVHeadend server. It’s also headless with only ftp, ftpd,telnet, telnetd and TVHeadend installed over PiOS Bullseye Lite. I can access TVHeadend remotely so that we can watch TV when we’re travelling in places where we have a 4G connection available on our phones and tablets but where there is no terrestrial TV reception. Our portable TV plays nicely with my iPad Mini 5 and it works very well indeed. Sometimes, when we have full coverage we can even watch HD TV although standard definition is plenty good enough.

The Pi-Zero is now my GPS disciplined time server with a Real Time Clock and that’s its only function. It runs on 32bit Bullseye headless.

So what’s next ?

I think the next step is to try to move my wife to a Pi. She’s seen how well the Pi-400 works as a desktop so now all I have to do is to wean her off Windows. I had in mind upgrading my Pi-400 to a Pi 4 8GB as I do a fair bit of memory hungry photo and video processing. The Pi-400 will suit my wife perfectly as she does proof reading and word processing and gets most of her work via e-mail. The only issue I can see is not having Outlook but I’ve found that Evolution is a workable substitute for Outlook. She also does a load of web based work using Firefox so PiOS with Firefox installed will work perfectly for her. The Libre Office Suite works perfectly with Microsoft format documents so there’ll be no compatibility issue there either.

As far as the rest goes I’m still not entirely happy with where I’ve got everything so I’ll need to think about that. I also need to streamline my storage. I have a lot of spinning rust hard drives so re-organising them is also a priority – some partitions are short on free space while others have heaps. Loads are very uneven so evening things out would be good.

For the server(s) and desktops I really, really want to get rid of wi-fi and for wired systems move to ethernet over power line. I don’t like wi-fi, never have. Wi-fi is useful for portable devices like tablets, iPads, phones, etc. but in my view has no place with non portable devices.

So they’re my plans, anyway. Will it all happen ? Dunno…. We’ll see.

 

 

A Raspberry Pi 400 joins the clan

Do I have enough Pi’s

Clearly the answer was a resounding “Yes”. I had a Pi 4 4GB, a Pi 4 1GB and a Pi Zero. The people at Pi HQ decided that I should spend a bit money and get a Pi 400.. I only ordered the Pi 400, the mouse and the power supply as I already have a load of SD cards and many HDMI cables and adapters. The release of the Pi 400 worked in nicely with what I want to do – ditch Windows and go entirely with Linux so I ordered one from Core Elctronics. A few days later and I had it my hot little (well big, actually) hands.

First steps

Well the first thing to do was to put the RaspiOS image onto a spare Sandisk 500GB SSD. While that was happening I connected the Pi 400 to a monitor and stuck my “Basic” SD card in it to check if I could just lurch into USB booting. No problems.

I removed the SD card and plugged the SSD in and turned it on. As expected it all just worked. The next thing to do was to get it updated so that was done – time consuming process so I had a coffee while it was happening.

Next on the list was to get my “home pages” opening automatically when Chromium starts which was accomplished in a trice. Looks good too, but I need to twiddle the font sizes a bit.

Next steps

Now that I had it up and running with the internet connected and the browser working I started working on my “required” software. Libre Office, an email client, working sound,etc.

Libre Office was a straight install via “apt install” and of course went perfectly. Next came an e-mail client. I played around with Claws for a while until I realised that it doesn’t play nicely with the way I like to do things – I have six email accounts and that seems to complicate things too much with Claws.

More research was done and I figured that Thunderbird, going by the reviews, ticks all of the required boxes. Again, a quick “apt install” and I had thunderbird installed. I configured the email accounts which was a quite easy process. Then came the hard part.

What a clunky, slow, bloated piece of software Thunderbird actually is. Sure it can do what I want but it can only do it in its own (long) time. Even Outlook on a slow old Windows box is faster and slicker. The search continues for a better email client that can deal properly with POP3/SMTP and IMAP without having a major hernia.

Because I tend to have a lot of browser tabs open all the time and the email client open as well as a large spreadsheet I decided to use a real swap file – a simple edit of  /etc/dphys-swapfile and that was done.

So how does it all go ?

Well the first impression is that it is a whole lot quicker than my, admittedly, old Windows 10 box. I have all of my Samba shares from my other Pi’s and my Windows box mounted without much trouble.

Sound via Bluetooth was a bit of a bear to get working but I stumbled on a post in the Raspberry Pi forums that suggested that trying Pulseaudio was worth a try. It worked OK on a quick test of my JBL GO 2. There are reports of distorted audio which I’ll investigate later.

So where am I at now ?

Well that’s simple really. I’ll have to go through my Windows 10 box and have a look at which programmes I really want and get them, or their equivalent, installed and configured.

I need to get a decent email client to replace Thunderbird. If any readers have suggestions please send them my way.

So far I’m impressed. The performance has exceeded my expectations greatly. The Pi 400 seems to be more than capable of performing the tasks that I want it to.

And last, but by no means least, it is a cheap solution to my aging Windows box problem.

So now I have :-

  • A Pi 4 4GB hosting this web site as well as being a media player and MySQL database server.
  • A Pi 4 1GB which is a NAS system with Samba and about 6TB of attached storage. It runs the RaspiOS 64 bit OS.
  • A Pi Zero which is my “messing about” system. It gets used for all sorts of stuff while I try and learn about python3.
  • A Pi 400 which will become my desktop solution when I finish it.

And, of course, here’s the mandatory photo of the Pi 400 in action

Raspberry Pi 400 in action.

 

 

Adding a Pi Zero to the Pi menagerie

A Pi Zero

So I already have a Pi 4 4GB and 1GB. The 4GB Pi is where this web site is hosted along with our stash of TV shows and movies and as well as serving up the web site we use it to play our shows via VLC and a 4K HD TV. That all works very well and it just sits there and does its stuff. It’s visible to the internet on ports 443 for the web server and 22 for maintenance when I’m not at home. It has 1TB or storage attached.

The 1GB Pi is our file store – backups of our Windows systems, our digital photos, iTunes music, our movie and show collection, etc. It is also a MySQL server for various databases that I keep for community group stuff. This Pi isn’t visible to the internet at all and has about 5TB or storage attached all SSD.

Both the 1GB and the 4GB Pi’s are running 64 bit kernels although the 4GB Pi is 32bit userland. The 1GB Pi is fully 64 bit. They both boot from USB SSD devices.

Both of those Pi’s have Samba running and they share various directories. The whole setup works very, very well.

When I first got the 1GB Pi it was supposed to be a sort of development and test system for other stuff I’m working on. That scenario lasted only a short time, however.

So what to do about a test system ? A Pi Zero of course. It doesn’t matter what I wreck – I can just make another SD card if I mess up too badly. So I ordered one along with a power supply, a HDMI adapter and a plastic case.

After a small hiccup within about an hour or so I had it in its case with the 128 x 32 OLED Display  from Core Electronics attached and working.

The hiccup was born of not paying attention. I imaged the 64bit PoOS to the brand new SD card and spent a bit of time trying to work out why it wouldn’t boot. I should’ve read what the Pi Imager was telling me and used the correct version of PiOS. You can take it from me that a 32bit PiZero can’t use the 64bit OS. Idiot….

The longest parts were making the SD card from the latest correct download and updating it once I had it booted.

Pi Zero with plastic case and 128×32 display

First impressions ? Geeze it’s small and for such a small package it’s surprisingly capable – much more so than my 80’s IBM Thinkpad running OS/2 Warp.

Now that I’ve got my bascic configuration up and running I’ve taken a copy of the SD card using the handy built in utility so I’ve got a basic fallback position should I mess things up irretrievably.

Next step is to make a start on a python3 script to control the fans on the other two Pi’s then there’s a couple of other MySQL things that I want to do with the community group database and that’ll be followed by some HAM radio stuff that I’ve been looking at for a while now.

 

 

An even bigger small display

Adafruit 240×240 display

So after a bit of success with the previous two Adafruit displays I figured that I’d try their 240×240 display so I ordered one and given the current COVID-19 induced postal delays it eventually arrived.

I wasted no time in disabling the script that runs the 128×64 display and plugging in the new one.

Getting it working with a sample Adafruit script took no time at all so I now had to write another Python3 script to run the bigger display. I worked out roughly what I wanted to put on it and in which orientation. With a decent font size and by tweaking the height and width parameters in the script I found that I had eleven lines to play with.

As ever my Python3 skills are a bit Neanderthal so it took a while and was really, really messy to start with. But at least it worked.

Once I set to work putting a bit of doco in the script and cleaning it up a bit I only needed an hour or so to make it look a bit respectable, to me at least, and to have it still work. It’s here for your perusal if you so desire.

It even looks fairly neat apart from the fan wires that are not connected any more due to the display taking the 5V pins. I’ll have to work out a way of sourcing the 5V from somewhere else although the fans aren’t really needed – the Pi 4 4GB usually runs at less than 60 deg C even when playing HD videos with VLC.

Adafruit Mini Pi tft 1.3-240×240

What it displays

After a bit of messing around I have it displaying the date, the time, CPU temp, CPU and memory usage, disk usage, swap usage, uptime, boot time and three lines for button presses, etc. I’ll probably mess around with these lines and the button callbacks some time in the future.

Anyway another success with an Adafruit display. If you have a look at the script that drives it you’ll no doubt see that my Python3 skills are lacking and that I’ve probably gone about everything in the most inefficient way possible.

There are a few bits of the script that I found as code snippets and messed about with to get the results that I wanted, other bits are directly from Adafruit.

For my next trick I want to have a slideshow running and have the buttons set up in such a way that I can switch what I’ve got now to a slideshow and back again. I was think to have both buttons present a sort of menu so that I can Poweroff, Reboot or go with a slideshow. I have absolutely how I’m going to do it yet though.

Stand by for future developments. In the meantime, if you have any suggestions for the script(s) that I’ve already written please let me know.

 

 

 

 

 

 

 

A slightly bigger display

The Adafruit 128×64 OLED Bonnet for Raspberry Pi

So after getting the little display I decided to have a go with the Adafruit 128×64 OLED Bonnet for Raspberry Pi. After a bit of messing about this is what it looks like in operation

Adafruit 128×64 OLED Bonnet for Raspberry Pi

I’ve put eight lines of text on it and enabled a few of the buttons / joystick. The test is date, time, cpu temp, CPU and memory used percentages, root partition usage, swap usage, uptime and the boot time. The buttons thus far assigned are button 5 for reboot, button 6 for power off, joystick down is Samba restart and joystick up is Apache2 restart. It required a fair bit of messing around as well as learning a bit about GPIO pins and acting on the buttons.

Anyway here’s the Python3 script. It isn’t particularly well commented – the comments were more for my use while I was writing it. And nor is there any error checking. Python3 seems to do a pretty good job of that.

The next step

For my next trick I want to have a play with a Adafruit Mini PiTFT – 135×240 Color TFT Add-on for Raspberry Pi. I don’t know what I’ll display on it yet but I’m sure I’ll think of something.

I’m also tossing around the idea of having a play with the PiTFT Plus 320×240 2.8 TFT + Resistive Touchscreen

Because of all this messing around during the COVID-19 restrictions that we are living under here in Victoria, Australia I reckon I may have to get another Pi. Probably a 2GB Pi 4 this time – I already have a 4GB and a 1GB and I cant see a reason for forking out the extra cash for the 8GB iteration.

Of course another Pi means another SSD to boot from, another case and probably a fan shim. Hmmm. I can see the price of all this creeping well toward unaffordable already.

 

 

A neat little display

A four line system monitor

Whilst doing a bit of idle web surfing I stumbled across this little gizmo and thought that it would provide me with a few hours of amusement so I went ahead and ordered one. After it arrived it sat on my desk for a few days while we shot off to Omeo for a bit of a break for a few days.

When we got back I installed it and started playing. Fortunately Adafruit have a sample python script which I was able to massage to come up with this.

Raspberry Pi 4 4GB with a heatsink case and four line display

As you can see it displays the date, time, CPU temp and memory and CPU usage.

I’ve already got USB booting happening so it boots from a 1TB Seagate USB 3.0 SSD which is just two partitions – “/boot” and “/”. That works well too.

The python script

This was, or is, my first ever python script. I’m adept at Perl, COBOL, Fortran, etc. so I was able to work out a simple script with only minimal messing around. It’s a good thing that Adafruit have some detailed instructions which include a python script to get it working. A bit of modifying / rewriting the Adafruit script and I had it displaying what I wanted.

So, without further ado here’s here’s the Python3 script which I was going to embed here but WordPress is wilfully destroying the indentation.  A quick look will reveal that there is no error handling whatsoever. I haven’t learnt how to do that yet so that’ll be introduced on another day. There’s also almost no commenting either.

 

What now

Well now that I’ve got this working and working well I thought that I’d get one of these which has buttons and is a bigger. The little one that I have now will go onto my other P1 4 (1GB) which is used for storage and as a test bed for this site. It’s also booting from USB and is also running Pi OS 64bit which seems to me to be nice and stable.

The bigger display will allow more lines of text but I don’t know what I’ll put there yet. It also has a couple of buttons and a joystick. I can use the buttons for things like “sudo reboot” or “sudo poweroff” or something.

Stand by for future developments.

 

 

 

USB Boot

Why ?

Why USB boot ? Well for a start I’m a tinkerer who just can’t leave things alone. Secondly I’ve had an SD card fail and didn’t want to go down that path again – maybe I should’ve got better cards ? And thirdly I’ve always, rightly or wrongly, regarded the SD card boot scenario a bit clumsy.

I got rid of some of the clumsiness by moving the root partition to a 1TB Toshiba X1 portable SSD. This was easily done and was a great success. I still had a bit of SD clumsiness though which I wanted to get rid of.

Then along came the announcement of USB booting and for once in my life I didn’t leap into the deep end with the first BETA. I waited until the BETA was elevated to STABLE and then leapt into the deep end.

What follows is what I did to get it working.

First steps

The first thing to do was to get my hands on another portable SSD.  This time it was a Seagate 1TB portable SSD. Not cheap but hey, you can’t let money stand in the way of a bit of tinkering, can you ?

The second thing to do was to make sure that my existing installation was completely up to date by doing “sudo apt update” followed by “sudo apt full-upgrade”. Because I hadn’t done this for a while there were about sixty packages updated. While I was at it I had a look to see what I could get rid of and removed a couple of packages that I didn’t require.

Next on the list was to update the boot loader and get it configured. That’s the bit the seemed a bit daunting at the start.

Update the bootloader

I was a bit worried about this bit but only because I had no real backout plan if it all went badly. I needn’t have worried.

First up I had a look at “/lib/firmware/raspberrypi/bootloader/” and saw that there was a “stable” directory and saw that there was “pieeprom-2020-06-15.bin” which was what I was after according to this topic on the Raspberry Pi forum.

Because I have a second Pi that USB boots 64 bit Raspbian that I set up a couple of weeks ago I had already done a lot of reading about the configuration. The story of the second Pi is here.

Anyway I’m jumping the gun a bit here. Before updating the bootloader I downloaded the latest version of Raspberry Pi OS and using the Raspberry Pi Imager I set my Seagate portable SSD up with it. While that was happening I got the bootloader updated and configured. Easily done in a few steps.

I needed a configuration so I used the one from my 64 bit Pi :-

[all]
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
DHCP_TIMEOUT=45000
DHCP_REQ_TIMEOUT=4000
TFTP_FILE_TIMEOUT=30000
ENABLE_SELF_UPDATE=1
DISABLE_HDMI=0
BOOT_ORDER=0xf41

I got this by doing vcgencmd bootloader_config > bootconf.txt on my 64 bit USB booting Pi. It was a simple matter to copy that to my non USB booting Pi.

Back to the 32 bit Pi and I did “rpi-eeprom-config –out pieeprom_new.bin –config bootconf.txt  /lib/firmware/raspberrypi/bootloader/stable/pieeprom-2020-06-15.bin”  which gave me a new bootloader called pieeprom_new.bin to install.

Installing the bootloader was my mental sticking point. Up to this point I had a working system with the root partition on an SSD and the /boot partition still on an SD card. Would it still boot after the new bootloader was installed ? I certainly hoped so!

Anyway I went ahead confident that I could roll it back to the old non USB booting model simply by installing the bootloader from the “critical” directory. Anyway I did a “sudo rpi-eeprom-update -d -f ./pieeprom_new.bin” followed by a reboot and it booted just fine. I did a “vcgencmd bootloader_version” and a “vcgenccmd bootloader_config” and was greeted by the new version and the new configutation. It had worked and my web server / samba / other stuff system worked as it it always did.

 

Making the new boot SSD

Now to the fun part.

Remember the SSD that I made with the latest Raspberry Pi OS ? Well now its time to shine has arrived.

I plugged it into the Pi and copied /boot/*.elf and /boot/*.dat to the boot partition of the SSD. I closed the Pi down (sudo poweroff and unplug the power) and unplugged the original SSD and the SD card leaving the new SSD plugged in. I plugged the power in and lo and behold it booted OK and I went through the whole setup process. That’s half the job done. The brand new SSD had the boot and root partitions and all was right with the world.

I powered off again and rebooted the original configuration – SD + SSD (Toshiba). The new Seagate was plugged in and I formatted the root partition “mkfs -t ext4 /dev/sdxx” and mounted the new empty, clean partition in /mnt. Then came “rsync -axv / /mnt” There’s about 700 GB in that partition so it was going to take a while so I went to bed.

One caveat here. If I plug both SSD’s into USB-3 ports it will fail at some point probably, I think, because it’s on the edge of the available power from a genuine Pi power supply. If I plug one of the SSD’s into a USB-2 port it’ll work OK so I did that which made it a lot slower.

It worked

The next morning, after a couple of cups of coffee, I had a look at what happened overnight. The “rsync” had completed without error.

At this point I needed to copy from the SD card config.txt and cmdline.txt to the new SSD so I did that. I also needed to edit cmdline.txt to make sure that the PARTUUID pointed to my new SSD. On the new SSD I needed to edit /etc/fstab to put in the new PARTUUID’s for the boot and root partitions.

At this point I was very hopeful but not certain that it would work out properly but I went ahead and powered the Pi down and removed the original SD and SSD and plugged the new SSD into a USB-3 port and plugged the power in. Success. Everything works OK. This web site works OK as does Samba and everything else.

The whole operation was a success. Apart from the “rsync” it only took about half an hour. Because of the sheer quantity of “stuff” the “rsync” was a lengthy affair.

The next step will be to update with Raspberry Pi OS 64bit when that becomes prime time. I’m sure that won’t be a simple upgrade like this ended up being.

Pi 4 USB Boot and 64bit Raspbian

Introduction

I have a perfectly working Raspberry Pi 4 1GB which is acting as a Samba server with a few TB of storage attached which we use to store our nightly backups, scanned and digital photos and a bunch of other stuff which is important to us. It was working well and didn’t need fixing or otherwise messing with at all.

Then the USB boot and 64 bit beta’s became available so I just had to try them out, didn’t I ?

Suffice to say that it all worked perfectly and now my Samba server has booted from a USB device and is running the new 64 bit beta OS.

Here’s how I did it.

USB Boot

All I did was to follow the instructions here. Please have a read of this most excellent doco before you do anything else.

Another thing that must be done before you start is to make sure that the proposed USB boot device works properly with your Pi. I have a couple here that work perfectly with Windows and iPad but just will not be recognised by Raspbian Buster.

Make a new SD card. I downloaded the image from the Raspberry Pi download page and put it on a new SD card using the Raspberry Pi Imager – Your mileage may vary if you use Balena Etcher or something else.

The next step is to put your image burner to use again and burn the image to the USB boot device that you will wish to boot from and put it to one side.

Boot from the new SD card that you just created and go through the setup process. Then do a “sudo apt update” followed by a “sudo apt full upgrade”. I rebooted the Pi at this stage. Next comes the first of the fun parts.

As root, edit /etc/default/rpi-eeprom-update and select BETA releases. I found that you need to have “beta” in lower case. Then do “sudo rpi-eeprom-update -d -f /lib/firmware/raspberrypi/bootloader/beta/pieeprom-2020-05-15.bin”

Do “rpi-update” and copy *.elf and *.dat from /boot to the boot partition on the USB device that you’ve put to one side. Do a “sudo poweroff” and remove the SD card. Reconnect the power and it should boot from the USB device if you’ve done it right.

I followed the instructions here and they proved to be invaluable. The whole operation worked properly the first time I tried it so follow the instructions in the doco and you’ll probably not go wrong. If it all goes wrong just go back to the beginning and start over.

I can still boot from my original SD card if I need to so I have a fallback if I need it.

64 bit Raspbian

While I was making my Pi boot from a USB device, Raspberry announced a 64 bit beta of Raspbian Buster. Of course, I just had to try it out in conjunction with USB booting.

I downloaded the 64 bit Buster image and using the Raspberry Pi Imager I prepared a USB SSD with it.

I did a “sudo poweroff” and swapped the USB devices over and plugged it back in. Imagine my surprise when it booted without any issues. I did the setup and I had a Pi with 64bit Raspbian Buster booted from a USB device. Not an SD card in sight..

Next was to get it updated and Samba installed. Again it went without a hitch. To make life a bit easier I copied fstab, /etc/dhcpcd.conf and smb.conf from my original SD card. “fstab” was not a direct replacement. I just edited the fstab (on the new 64 bit device) and put the mounts for my storage in it. The other two files were direct replacements though. Rebooted and it looks exactly the same as it did with the original SD card.

And it all came together

Well, at the end of the processes it all works perfectly. I’m sure that there will be more issues to overcome as the USB boot and 64 bit Raspbian move out of “beta” and into “production”

In the grand scheme of things it gained me exactly nothing apart from a bit more knowledge about the workings of the Pi and Raspbian.

I’m not touching the Pi that this web site is on – it’s a web server, a media player and store, a database server, etc. This Pi I regard as being critical and all precautions are taken to keep it up and running. I take nightly incremental backups and weekly full backups of the whole system including MariaDB database backups.

Sooner or later though, I’m going to have to bite the bullet and fully upgrade this Pi – a prospect that I’m most certainly not looking forward to.

LAMP+WordPress

Seeing as I managed to get USB boot and 64 bit Raspbian working so easily I figured that I may as well go for broke and get a duplicate of this web site happening.

First up I installed Apache, MariaDB, php and php-mysql. That was all very easy as it usually is.

Next up was to make a database for WordPress and create the relevant user. No problems here either. Then I installed the latest copy of WordPress and configured it. Again, no problems.

The bit that I expected the most trouble from was getting a copy of the WordPress installation and data across from this site to the new Pi configuration. All I did was to install the duplicator plugin on both sites and export it from this site and install it on the test Pi.

So now as well as Samba, I have LAMP+WordPress working. It’s hardly the definitive test, I know, but it’s an encouraging preview for when 64 bit Raspbian comes out of beta and into the real word.

Another good thing to come out of this exercise is that I may have discovered a path towards upgrading when USB boot and the 64 bit OS comes into prime time.

The importance of backups

What happens when it all goes pear shape

A couple of days ago we had a small power “glitch” here. Apart from the microwave clock going back to zero and the TV going off the Raspberry Pi that this site runs on went down.

When I powered it back up I found that my WordPress site was devoid of content. Further panic stricken investigation revealed that the “wp_posts” table in the database had a corrupt index. After much gnashing of teeth, googling and experimenting (after taking a copy of course) I was unable to fix the problem.

What to do ? Well I bit the bullet and restored a backup I’d take about a week earlier using the “Updraft” plugin. Apart from losing a week’s worth of content all was well. And this is where good fortune stepped in.

During the couple of hours before the power glitch I’d been messing around putting an “RSS Subscribe” in the side bar and testing it on my iPhone. It proved to work well too so on a whim I had a look at the RSS on my phone and it had handily retained all of the content from this site so I was able to e-mail it to myself. From there it was a simple matter to copy and paste the content to the recreated posts. Whew.

The only thing that I lost was about a week’s worth of stats from “WP Statistics” plugin. I can live with that but I’d rather that I didn’t have to though.

OK so I was lucky

So, this time I was lucky ! I only lost a very minimal amount of stuff. The stats and the configuration for Google AdSense.

As a retired large system administrator I know full well the importance of good backups. I wasn’t taking them on a regular basis because “what could possibly go wrong”.

I’m now using a Linux backup package for the whole Pi and I’ve scheduled a backup of WordPress once a day. As well as that I’m taking an Updraft backup right atfer I do anything of any note to WordPress or the system as a whole.