Updated transmission image

 Simple base image update to alpine 3.15.1, no other changes.

flexget and finance-quote

My recent patch fixing a crash (undefined var) on anilist plugin of flexget when yuna.moe is unreachable has been merged. It's included on flexget 3.3.1 and on my docker image grmontesino/flexget:3.3.1-i1

In similar news, the Tesouro Direto finance-quote module I've submited to finance-quote also have been merged and should be available on finance-quote 1.52 release planned to mid-year.

WSL tricks part 2: VPN and MTU Setting

The first time you try to ssh to a remote server from WSL over a VPN connection, there's a good chance the connection will hang as soon as you start using it. After a little testing you may notice that it happens when using any command with longer output.

It turns out that the interface for the HyperV VM used to run wsl2 doesn't set its MTU correctly, and the network packets get discarded somewhere along the way. To fix this (assuming we don't have control over the VPN configurations) we need to set the MTU manually.

To find out what MTU setting should be used, the "tracepath" command can be used. Just install the package if needed, run "tracepath <server reachable by VPN>" and check the lowest "pmtu" value shown, that should be set as interface MTU.

For the configuration itself, a quick search on the internet did wield a few alternatives, but the nicer ones unfortunately didn't work... Setting the MTU directly on the HyperV configuration didn't get persisted across reboots; WSL's ubuntu doesn't seem to read any interface configuration on startup by default (and I don't see any good reason to install a init system on WSL)... 

In the end I've resigned myself and adopted the ugly solution: put the command to set it directly on ~/.bashrc:

#MTU
sudo ip link set eth0 mtu 1350

To get it at least a little less ugly, we can make it so sudo doesn't ask for a password: run "sudo visudo -f /etc/sudoers.d/mtu" and create the file with the following content:

ALL    ALL = (root) NOPASSWD: /sbin/ip link set eth0 mtu 1350

That will fix the hanging SSH problem. It's ugly because the MTU should be set dynamically / automatically and It's ugly because bashrc isn't the proper place for this kind of configuration, but at least it works.

Docker Image: BURP BackUp and Restore Program

Just released my images for burp on github. This is a somewhat more complex stack with 4 images - base, server, client and web user interface - and a bit more of scripting on the entrypoints to support the configuration file schema used by burp.

With 4 images in one repository and a dependency relation on the base for the other three, I wasn't sure how it would be to build everything automatically through github actions, but it turned out to be pretty easy. Just build base first, the other three after in parallel. 

Github actions matrix strategy option for running similar jobs with a few parameters changes with no need to make various almost identical copies of the definition and/or having to make some more complex reusable job configuration turned out to be very handy.

WSL tricks part 1: Windows' openssh key handling

For a long time I've been using a very simple setup on Windows to manage Linux servers: just plain old putty, pageant, winscp and the like. With the recent implementation and advances on WSL, I felt it was time to test using WSL to have a better environment, and it has been a very good experience so far.

As this setup required a few tinkering around to get everything into place, I'll be writing up a few of the thoughts and tricks I've gathered along the way. For start, let's talk a little about something which caught me by surprise: the native Windows' OpenSSH ssh-agent key handling.

The first piece of our small puzzle to make a Windows station a good administration platform for Linux servers is to have ssh connection to those servers. Windows terminal is a good start, being a tabbed multi-terminal with support for cmd, powershell and WSL.

After that, it was a very interesting to discover there's a native port of OpenSSH for Windows available. At first sight, this could be an interesting tool to get this level of access without even needing to use WSL. But there's a catch...

To use ssh, you'll be willing to load your ssh key over an agent so you don't have to type your super-secret-long-passphrase every time, having it work automagically during your work session. The concept around ssh-agent is that your decryptographed key will stay in memory, never touching disk / permanent storage, and as such will be securely erased when you unload it and/or power down (assuming no one is going to do some crazy ninja cold RAM data extraction to get your ssh-key).

By the other hand, Windows' openssh port just disregarded the whole concept and instead permanently store your ssh key on Windows Registry, which in turn get persisted on disk itself. While it seems to be cryptographed over some kind of user-context security, it didn't really strike me as particularly safe, and there's some article somewhere explaining how it could be recovered by someone with access and how it would be very hard to get it fully out of disk once it gets there.

While I didn't dig much deeper in this, what I saw was enough to convince me to stay away from windows' openssh - there certainly seems to be some risk, and wsl + ubuntu ssh works perfectly fine for my needs. 

finance-quote development and Tesouro Direto

Recently sent a pull request for the inclusion of the Tesouro Direto module on finance-quote, which is the library used by gnucash for "commodities" quotes. If that goes well I might be sending one or two more modules I've made (or want to make) and use (or want to use).

Also published a simple development environment container with the fq's dependencies pre-installed - pretty nice way of developing and testing things without installing lots of dependencies directly and risking messing the "stable" version in "production" use.

docker-flexget 3.2.18-i1

Released docker-flexget 3.2.18-i1 yesterday. 

Simple update to keep up with upstream (which already includes my smallish patch to fix some problems introduced to anilist client on 3.2.5).

Incremental pvmove

I've recently had some fun with a BTRFS filesystem corruption (described in a gist here). Ended up with the recovered data in a linear LV formed by 2 non-redundant PVs.

After all needed checks were done and a fresh backup taken, it was time to eliminate the failed btrfs filesystem and get the data back on redundancy by reusing its RAID5 array. In my mind, it would be simple a question of pvmove'ing the data, which I could do over time, no hush, by making partial/incremental data moves.

Contrary to what I expected however, it turned out I couldn't just run pvmove for some time, cancel it and have the data which was already moved stay on the new volume; instead, after stopping pvmove the lvm structure stayed as it was at the start.

It turns out pvmove basis for data movement is the segment, that is, contiguous allocations of data, not individual extents as I expected. For the data movement to be "commited" it would have to finish moving the whole segment, which in this case would mean moving the entire PV / disk partition, as it's fully allocated to the same LV.

As pvmove also supports specifying specific areas of the PV to move, It seemed right to script my way to the result I wanted, so I've made a small script to sequentially pvmove a disk in "small" [parameterized] amount of extents per time. The script can be interrupted and the fully finished pvmove's stay as they should, so you can get back another time and continue from (mostly) where you stopped.

Please keep in mind that this has been made and tested in a very specific case - PV allocated sequentially to a single LV. I didn't really try to cover other user cases, so it could move data it shouldn't, and while I don't expect a pvmove to cause data loss, if the script somehow gets the start extent or size wrong, I can see it getting the LV / filesystem fragmented. As usual, having a backup ready is recommended.

Without further ado, my hackish incrementa_pvmove.sh script is available here: 

https://gist.github.com/grmontesino/8ec29cd16cf3d893dde808f35f079304

Yuube - last night / evening

After some time around non-native languages, something which you'll notice is that they've what seem like "traps" designed to make confusion and/or cause miscommunications. 

I guess our native languages do have these traps too, but being used to them we don't even notice - in the end of the day, on real world usage the right meaning ends up being clear for context (or we just miscommunicate anyway and life follows).

Today's trap from my morning Japanese goes to "yuube" - the same word (or two words with the same pronunciation?) has two meanings dangerously close but different to each other:

昨夜　last night

夕べ     evening

https://jisho.org/word/%E5%A4%95%E3%81%B9

Docker images: Transmission and Flexget

 Hello,

Just a quick note that I've published (after some cleanup and improvements) the docker images I use for transmission (bittorrent client) and flexget (media management automation). The "source" for the images, along with documentation and suggested docker-compose files, are available at github:

• https://github.com/grmontesino/docker-transmission
• https://github.com/grmontesino/docker-flexget

As a fun learning project, github actions have been configured to automatically build the respective images and automatically push then to Docker Hub:

• https://hub.docker.com/repository/docker/grmontesino/transmission
• https://hub.docker.com/repository/docker/grmontesino/flexget

While those are simple images perfect for some testing, I expect to publish some of the more complex stuff I use in the future.

I am back (?)

Just found out this blog still exists and had some fun reading the posts from 10+ years ago... Some things changed so much, others are still exactly the same...

Maybe I can find some inspiration to make some more brain dumps around and document a bit of what I've been doing lately; who knows...