Updated transmission image

 Simple base image update to alpine 3.15.1, no other changes.

flexget and finance-quote

My recent patch fixing a crash (undefined var) on anilist plugin of flexget when yuna.moe is unreachable has been merged. It's included on flexget 3.3.1 and on my docker image grmontesino/flexget:3.3.1-i1

In similar news, the Tesouro Direto finance-quote module I've submited to finance-quote also have been merged and should be available on finance-quote 1.52 release planned to mid-year.

WSL tricks part 2: VPN and MTU Setting

The first time you try to ssh to a remote server from WSL over a VPN connection, there's a good chance the connection will hang as soon as you start using it. After a little testing you may notice that it happens when using any command with longer output.

It turns out that the interface for the HyperV VM used to run wsl2 doesn't set its MTU correctly, and the network packets get discarded somewhere along the way. To fix this (assuming we don't have control over the VPN configurations) we need to set the MTU manually.

To find out what MTU setting should be used, the "tracepath" command can be used. Just install the package if needed, run "tracepath <server reachable by VPN>" and check the lowest "pmtu" value shown, that should be set as interface MTU.

For the configuration itself, a quick search on the internet did wield a few alternatives, but the nicer ones unfortunately didn't work... Setting the MTU directly on the HyperV configuration didn't get persisted across reboots; WSL's ubuntu doesn't seem to read any interface configuration on startup by default (and I don't see any good reason to install a init system on WSL)... 

In the end I've resigned myself and adopted the ugly solution: put the command to set it directly on ~/.bashrc:

#MTU
sudo ip link set eth0 mtu 1350

To get it at least a little less ugly, we can make it so sudo doesn't ask for a password: run "sudo visudo -f /etc/sudoers.d/mtu" and create the file with the following content:

ALL    ALL = (root) NOPASSWD: /sbin/ip link set eth0 mtu 1350

That will fix the hanging SSH problem. It's ugly because the MTU should be set dynamically / automatically and It's ugly because bashrc isn't the proper place for this kind of configuration, but at least it works.

Docker Image: BURP BackUp and Restore Program

Just released my images for burp on github. This is a somewhat more complex stack with 4 images - base, server, client and web user interface - and a bit more of scripting on the entrypoints to support the configuration file schema used by burp.

With 4 images in one repository and a dependency relation on the base for the other three, I wasn't sure how it would be to build everything automatically through github actions, but it turned out to be pretty easy. Just build base first, the other three after in parallel. 

Github actions matrix strategy option for running similar jobs with a few parameters changes with no need to make various almost identical copies of the definition and/or having to make some more complex reusable job configuration turned out to be very handy.

WSL tricks part 1: Windows' openssh key handling

For a long time I've been using a very simple setup on Windows to manage Linux servers: just plain old putty, pageant, winscp and the like. With the recent implementation and advances on WSL, I felt it was time to test using WSL to have a better environment, and it has been a very good experience so far.

As this setup required a few tinkering around to get everything into place, I'll be writing up a few of the thoughts and tricks I've gathered along the way. For start, let's talk a little about something which caught me by surprise: the native Windows' OpenSSH ssh-agent key handling.

The first piece of our small puzzle to make a Windows station a good administration platform for Linux servers is to have ssh connection to those servers. Windows terminal is a good start, being a tabbed multi-terminal with support for cmd, powershell and WSL.

After that, it was a very interesting to discover there's a native port of OpenSSH for Windows available. At first sight, this could be an interesting tool to get this level of access without even needing to use WSL. But there's a catch...

To use ssh, you'll be willing to load your ssh key over an agent so you don't have to type your super-secret-long-passphrase every time, having it work automagically during your work session. The concept around ssh-agent is that your decryptographed key will stay in memory, never touching disk / permanent storage, and as such will be securely erased when you unload it and/or power down (assuming no one is going to do some crazy ninja cold RAM data extraction to get your ssh-key).

By the other hand, Windows' openssh port just disregarded the whole concept and instead permanently store your ssh key on Windows Registry, which in turn get persisted on disk itself. While it seems to be cryptographed over some kind of user-context security, it didn't really strike me as particularly safe, and there's some article somewhere explaining how it could be recovered by someone with access and how it would be very hard to get it fully out of disk once it gets there.

While I didn't dig much deeper in this, what I saw was enough to convince me to stay away from windows' openssh - there certainly seems to be some risk, and wsl + ubuntu ssh works perfectly fine for my needs. 

finance-quote development and Tesouro Direto

Recently sent a pull request for the inclusion of the Tesouro Direto module on finance-quote, which is the library used by gnucash for "commodities" quotes. If that goes well I might be sending one or two more modules I've made (or want to make) and use (or want to use).

Also published a simple development environment container with the fq's dependencies pre-installed - pretty nice way of developing and testing things without installing lots of dependencies directly and risking messing the "stable" version in "production" use.

docker-flexget 3.2.18-i1

Released docker-flexget 3.2.18-i1 yesterday. 

Simple update to keep up with upstream (which already includes my smallish patch to fix some problems introduced to anilist client on 3.2.5).

Incremental pvmove

I've recently had some fun with a BTRFS filesystem corruption (described in a gist here). Ended up with the recovered data in a linear LV formed by 2 non-redundant PVs.

After all needed checks were done and a fresh backup taken, it was time to eliminate the failed btrfs filesystem and get the data back on redundancy by reusing its RAID5 array. In my mind, it would be simple a question of pvmove'ing the data, which I could do over time, no hush, by making partial/incremental data moves.

Contrary to what I expected however, it turned out I couldn't just run pvmove for some time, cancel it and have the data which was already moved stay on the new volume; instead, after stopping pvmove the lvm structure stayed as it was at the start.

It turns out pvmove basis for data movement is the segment, that is, contiguous allocations of data, not individual extents as I expected. For the data movement to be "commited" it would have to finish moving the whole segment, which in this case would mean moving the entire PV / disk partition, as it's fully allocated to the same LV.

As pvmove also supports specifying specific areas of the PV to move, It seemed right to script my way to the result I wanted, so I've made a small script to sequentially pvmove a disk in "small" [parameterized] amount of extents per time. The script can be interrupted and the fully finished pvmove's stay as they should, so you can get back another time and continue from (mostly) where you stopped.

Please keep in mind that this has been made and tested in a very specific case - PV allocated sequentially to a single LV. I didn't really try to cover other user cases, so it could move data it shouldn't, and while I don't expect a pvmove to cause data loss, if the script somehow gets the start extent or size wrong, I can see it getting the LV / filesystem fragmented. As usual, having a backup ready is recommended.

Without further ado, my hackish incrementa_pvmove.sh script is available here: 

https://gist.github.com/grmontesino/8ec29cd16cf3d893dde808f35f079304

Docker images: Transmission and Flexget

 Hello,

Just a quick note that I've published (after some cleanup and improvements) the docker images I use for transmission (bittorrent client) and flexget (media management automation). The "source" for the images, along with documentation and suggested docker-compose files, are available at github:

• https://github.com/grmontesino/docker-transmission
• https://github.com/grmontesino/docker-flexget

As a fun learning project, github actions have been configured to automatically build the respective images and automatically push then to Docker Hub:

• https://hub.docker.com/repository/docker/grmontesino/transmission
• https://hub.docker.com/repository/docker/grmontesino/flexget

While those are simple images perfect for some testing, I expect to publish some of the more complex stuff I use in the future.

Friday 13th; Unix Time: 1234567890

Tomorrow will be Friday 13th. To make this the coolest friday 13th ever, at 23:31:30 UTC (in São Paulo daylight savings time that would be 21:31:30), Unix Time¹ will be "1234567890". Cool, isn't it ^_^

Thanks to that, I've seen what might be the coolest programing joke I've ever seen on the slashdot history comments:

Every day is a long day, because 86400 seconds won't fit in a short."² — FooAtWFU,

---

1: Unix systems keep the time by storing the amount of seconds which elapsed since midnight of January 1st, 1970.

2: a (unsigned) short in programming is a variable which can keep a value from 0 to 65525 (aka 2¹⁶-1).

HP Laserjet P3005d

We have acquired a new HP Laserjet P3005d printer at work some months ago, to replace an aging HP laserjet 2300dn which was consistently jamming on duplex jobs. The new printer just doesn't feel as good as the older one though... Anyway, there are some notes about problems I've found with it for reference.

CUPS, driver and advanced options/booklet printing

The printer was conected by USB to a Debian GNU/Linux server. With the 2300, running the setup allowed one to choose "client setup", tell the URI of the printer and have it just work out of the box with all the resources enabled. Unfortunately, this option isn't available in the 3005 driver (maybe HP people want to sell more network modules for the printers?).

To work around this issue while getting advanced options, such as booklet, working, one needs to install the printer as if it was connected on the computer, and create a new "local port" with the printer samba URI as "name" when asked about the printer port. (If such advanced options aren't needed, following the steps on the samba howto to install the printer driver on the server works well).

Strange printing anomalies after changing the toner

After replacing the toner with a new one, there was some horizontal bars and strange bubble-like stains on the printings. Took some time to figure out that the basic mandate of the computing would solve this one: just turn the printer off, give it some small time and turn it back on.

How to NOT make a web form

The 2009 trainee's program of the multinational Unilever are open. As I've graduated recently, and heard some people say very good things about employment there, I've decided to apply myself.

With that in mind, I've followed the links on the Unilever page to the register myself. This brought me to an interesting web-based form, apparently designed by Cia de Talentos, an recruitment agency around there: Instead of using standard html selection boxes (ie, select elements), they just got creative and decided to reinvent the well using input elements and some "nice" down arrow images, on which you can click to get a "lovely" popup on which you can select you choice, which then gets dutifully copyed back to the input. Of course, they also set the choice's code on an hidden input element, presumably to easy their parsing later... Oh, and some of them even have the proper code, using select element, commented above them (for easing of legibility, I suppose...)

Now that I've let everyone anxious to see such beautiful art of the web programming, I'll let you see a snippets of the code:

<input type="text" name="Text_CodPais_cand" size="30" value="" onFocus="SetSav('SavRegioes'); SetSav('SavCand'); mudaFoco('Nome_cand',-1)" style="color: #0000FF; background-color: #EEEEEE; font-size: 8pt; font-family: verdana; width: 260px"> <img src="img/SetaCombo.gif" align="absmiddle" onClick="SetSav('SavRegioes');SetSav('SavCand');setCmb('un país', 'CodPais_cand',-1,280,260,0,'Nome_cand',-1)"> <input type="hidden" type="text" name="CodPais_cand" value="31">

I haven't really parsed all of their javascript (it's long, messy...), but it seems to make some "clever" stuff like create the full html of the popup using document.write.

They also seem unable to keep track of your language around the multipage form, so I also had to guess some Spanish stuff to be able to fill it fully (and got wondering in what language I should fill some fields also... settled on keeping everything in Portuguese.

To make this long history short, the main lessons I would take from this on the top of my head:

• Never ever make a fake select by using input and img. If you think it's the solution to your problem You're doing it wrong. Trust me, You'll look foolish, and some moron (or not) with so much time (or not) will make sure to tell the whole, lazy web on his blog.
• Keep the language of your web pages consistent.
• Be careful if you're going to outsource something. Some times people might notice it wasn't you which has done the foolish thing, some times not. Independently, they might chose to do business with you, or might think you aren't worth it if you can't even get a simple webpage done right. Either way it will damage your image if someone does something so messed up in your name.

Just for the record, in this case in particular I've chosen to register myself anyway; as I've got some really good references about Unilever. But I'm worried if this is the competence to be expected from their IT sector (which is also the sector I would work) - even if this wasn't their doing, there should be someone watching the quality of their outsourced work...

Hardware failures...

The Bad:

When I got to work today, I've found our server (white box with Debian etch responsible for networking, files, printers, etc) powered off - most likely the UPS' battery didn't survive a power outage during the night. When I turned the server on I was greeted by all those nice lines telling me I had a hard disk problem.

The Ugly:

Instead off marking the disk showing the read errors as bad, the RAID stack (device mapper?) somehow concluded the "good" disk of the RAID 1 array wasn't synched and kicked it out...

The good

The bad sectors did take only some unimportant collectd status files with them. After some poking with dd trying to force the HD to redirect the bad sectors, the read errors vanished and the Reallocated Sector Count didn't increase according to smartctl, which seems like a good signal.

Moving and resizing partitions

Earlier today I've migrated all the data from a 80 GB HD to a new one of 250 GB. It was a pleasing surprise to see how far the support for these kinds of things have gone.

The old HD had 4 partitions: The first two for Windows XP on NTFS (not my computer...), the third a small linux swap and the last a ext3 root partition for ubuntu.

The process was quite simple: Set the jumper on the new (samsung) HD, as the motherborad has one of the VIA chipsets without support for the 3Gbps models, connect both of them, boot with an ubuntu live CD, dd everything from the old HD to the new one, disconnect the old HD, boot with the live cd again and finally expand the partitions on the new HD using gparted.

The big surprise came in the last step above: I really didn't expect resizing NTFS partitions through Linux to work, and it went flawless (actually, I had a small glitch on resizing the ext3 filesystem, which forced to run resize2fs by hand with the force [-f] flag, as it kept asking for fsck).

More RAM 2

Note to self: Whenever adding RAM, besides looking for the correct count on POST, it seems to be a good idea to see also on kernel's boot logs if the kernel found and can use all the available memory.

In this case, I had to recompile the kernel with HIGHMEM enabled. Well, the kernel I had running (2.6.18) was starting to get old anyway, so now I've got 2.6.24.7, which seems to be the latest one on which the nvidia binary drivers will compile.

More RAM

Just added 1GB of RAM on my desktop, which had been crawling with 512MB; just 'cause there should be at least one good news even on bad days.

Hopefully Eclipse will be usable now.

Dear Itautec,

Please don't glue the processor on the heathsink with tape.

Thanks.

PS: If you ever get something like this, take the processor/heathsink out of the socket before trying to separate them, or you might end up pulling the processor out of the socket with it locked (like I did), which means a lot of bent pins.

PPS: It was an ancient computer (Itautec Transglobe 1001; AMD K6-II 450), they most likely don't do it anymore - I guess it would mess the heat dissipation on modern processors. Anyway, I don't think it was a great idea back then also...