Off-line migration of QEMU/KVM virtual machine

Assumptions

You have 2 computers, both with Linux (Linux Mint in my case). On one computer you have a QEMU/KVM virtual machine and you want to copy/move it to the other computer. On this second computer you have already QEMU/KVM installed and ready. You want to follow an off-line migration – so the virtual machine is stopped first, then recreated on the second computer. And then you can start it again on the second computer.

Solution

  1. Stop the virtual machine on the first computer.
  2. Locate the disk image file (like .qcow2 file) used by the virtual machine on the first computer. The Virtual Machine Manager program can help here. The file will be large (usually some GBs) but you need to copy it (somehow) to the second computer.
  3. Export virtual machine definion to an XML file:
    virsh dumpxml VMNAME > my_vm.xml
    
  4. Copy the XML file to the second computer. Edit the file to update path to the disk image. Search for a tag <disk>. It should contain a tag <source> having “file” attribute – you need to update its value so it points to the disk image file copied in step 2.
  5. If the VM was attached to a custom defined networks, there are some more steps – see: https://serverfault.com/questions/434064/correct-way-to-move-kvm-vm
  6. On the second computer run:
    virsh define my_vm.xml
    
  7. Run the Virtual Machine Manager on the second computer. It should show a new virtual machine that you have just imported. Run it.

In my case the first run failed with error message “the CPU is incompatible with host CPU: Host CPU does not provide required features: xop, fma4, tbm”. I’ve solved the issue by going to the tab with virtual machine details (in Virtual Machine Manager), then going to the “Processor” section and clicking checkbox “Copy host CPU configuration”:

You can find some more information here: https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-libvirt-config-gui.html#id-1.12.4.8.9.5

Then I’ve started my virtual machine again and it was fine!

Posted in Linux | Tagged , , | Leave a comment

How to install KVM/QEMU on Linux Mint 20.1

From time to time I want to run a virtual machine on my computer, a sandbox containing another operating system with some programs running in total isolation. Under Linux my answer for this need is a set of three components:

  1. KVMa virtualization module in the Linux kernel that allows the kernel to function as a hypervisor. (Wikipedia)
  2. QEMUa machine emulator and virtualizer that can perform hardware virtualization. It can cooperate with KVM to run virtual machines at near-native speed (Wikipedia)
  3. Virtual Machine Manager – a nice GUI to use above things as simply as possible.

Problem

How prepare all above components on a fresh installation of Linux Mint 20.1?

Solution

This solution is based on the article “Install KVM Virtualization on Linux Mint 20” with my additions.

First execute following commands:

sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager
sudo adduser $USER libvirt
sudo adduser $USER kvm
sudo adduser $USER libvirt-qemu

Here the above-mentioned article claimed everything is ready and working. But it wasn’t in my case. So just restart Linux now. Then proceed with verification steps:

virsh -c qemu:///system list

The output should be:

 Id   Name   State
--------------------

Then execute:

systemctl status libvirtd.service

The output should start with following lines:

โ— libvirtd.service - Virtualization daemon
     Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2021-02-03 22:54:41 CET; 31s ago

Then start Virtual Machine Manager. It should show that it is connected to KVM/QEMU. It should look like this:

Now it’s ready.

Posted in Linux | Tagged , , , | 1 Comment

How to install Wine 6.0 on Linux Mint 20.1

From time to time one needs to run a program, that was made for Windows – like a video game my son got (“Lego the Hobbit”), but one doesn’t want to run Windows at all. Still, this is possible with Linux thanks to the Wine project.

Problem

The problem is that repositories for Linux Mint / Ubuntu tend to offer quite old (outdated) version of Wine. For Linux Mint 19 it was version 3.x or 4.0 of Wine while the stable version claimed by Wine HQ at that time was 5.0. Now the stable version of Wine is 6.0 and Linux Mint 20.1 has only Wine 5.0 in its repositories. So to have newer version one has to install it in a little more complex way… like 7 bash commands. ๐Ÿ™‚

Solution

Use these instructions, based on “Install Wine 6.0 in Ubuntu 20.04 & Linux Mint” article, but corrected and verified by myself on a fresh installation of Linux Mint 20.1 64-bit (with Cinammon). First execute following commands:

sudo apt-get install libgnutls30:i386 libldap-2.4-2:i386 libgpg-error0:i386 libxml2:i386 libasound2-plugins:i386 libsdl2-2.0-0:i386 libfreetype6:i386 libdbus-1-3:i386 libsqlite3-0:i386
sudo dpkg --add-architecture i386
wget -nc https://dl.winehq.org/wine-builds/winehq.key
sudo apt-key add winehq.key

Now there is a step which is depending on the version of Linux. The command below, with word “focal”, is for Linux Mint 20.x:

sudo apt-add-repository 'deb https://dl.winehq.org/wine-builds/ubuntu/ focal main'

For Linux Mint 19.x you need to replace “focal” by “bionic” and for Linux Mint 18.x by “xenial”. For these older Linux Mint versions (19.x, 18.x) at this step you need to install “libfaudio0”: following the instruction from WineHQ this requires to download 2 files: libfaudio0_19.07-0_bionic_i386.deb and libfaudio0_19.07-0_bionic_amd64.deb and installing them with help of GDebi.

Then execute following commands:

sudo apt update
sudo apt install --install-recommends winehq-stable

And voila! The latest stable Wine is ready. I’ve verified the procedure on 2 computers, on Linux Mint 20.1 and Linux Mint 19.3.
The result: the above-mentioned video game was installed without problems from the original DVD and we were able to play it without issues. ๐Ÿ™‚

P.S.

A month ago, in December 2020, it was precisely 10th anniversary of this blog! ๐Ÿ˜€ Wow! 10 years passed since my first blog post. Thanks for reading! ๐Ÿ™‚

Posted in Linux | Tagged , , | Leave a comment

Problem with Logitech USB headset: can’t hear voice

This is the most ridiculous headset issue I’ve ever had: with a brand new USB headset from Logitech, model 960, after some month of normal operation a problem appeared such that one can no longer hear voice, but it was OK to hear music. Yes! One could hear music but generally without hearing any voice!!!

The issue was verified with different software (web browsers, Skype, MS Teams, etc.) on different computers, with Linux and with Windows. Unconditionally: fine with music, but no voice!

As having education with some electronics I’ve started to believe a sound amplifier built into the headset got broken and stopped amplifying some part of the bandwidth. Exotic, right?

The solution

The solution was as simple (and ridiculous!) as just changing left-right balance from the default, neutral position.

All credits should go to the author of the “How To – Fix the Sound on the Logitech USB Headset!” video (published in 2016!), which is presenting how to fix this issue under MS Windows (setting one channel, left or right, to 0% and the other one to 100%).

Under Linux this usually looks different – you need to change the left/right balance slider to a position other than the center using a sound control panel like mate-volume-control:

Actually I recommend setting the slider position somewhere near the center. After this simple step the headset started to work as expected! Voice is back. One can hear music as well. I believe Logitech is to blame.

P.S.

This is published in “Linux” category but the issue itself was present on Windows as well.

Posted in hardware, Linux | Tagged , , , | Leave a comment

Special characters in a key in Spring Boot YAML file

To use some special characters in a key of a property in a YAML file (like application.yml) processed by Spring Boot 2.0 (or later) you need to use specific syntax. Key element containing โ€œ/โ€ or โ€œ@โ€ or โ€œ+โ€ (and possibly some other special characters) needs to be surrounded by square brackets. Example:

foo.bar:
  "[http://something]": value

This is needed to define property named "foo.bar.http://something" in Spring Boot 2.0 or later. According to Spring Boot issue #13404 (Allow map binding to work with unescaped characters) it was not needed in Spring Boot 1.5 (and most likely in earlier versions).

If you don’t use square brackets in such a case when using Spring Boot 2.0 or later the actual property name will be different causing things like value injection (@Value annotation) to not work.

Another Spring Boot issue, #14017 (Property binder does not allow special characters in map keys), explains that with Spring Boot 2.0 (or later) special characters are dropped silently when square brackets are not used.

It seems this feature was not document well for some time – see Spring Boot issue #13506 (Document when and how to use bracket notation when binding to a map).

P.S.
I’m not claiming it is a good idea to use such characters as “/”, “@” or “+” in property keys with Spring Boot. I recommend to avoid such design decisions as it may make it really hard to override properties using for example environment variables.

Posted in Java, Spring | Tagged , | 1 Comment

Installing PlayOnLinux under Linux Mint 19.3

Here are commands needed to install PlayOnLinux together with its main dependencies:

wget -q "http://deb.playonlinux.com/public.gpg" -O- | sudo apt-key add -
sudo wget http://deb.playonlinux.com/playonlinux_bionic.list -O /etc/apt/sources.list.d/playonlinux.list
sudo apt-get update
sudo apt-get install playonlinux
sudo apt-get install multiarch-support
sudo apt install --install-recommends wine-installer
sudo apt-get install xterm

NOTE 1: Thanks to first 2 lines we’re getting latest stable version of PlayOnLinux (which was 4.3.4 at the time of writing) instead of some older version available in the repository (some 4.2 at the time of writing).

NOTE 2: the 2nd step is specifically adjusted for Linux Mint 19.3. Please read “Ubuntu” section on PlayOnLinux Downloads page to find correct command version for your version of Linux Mint or Ubuntu.

After these steps I was able to start PlayOnLinux and use it to install “Enter the Matrix” game (it was officially made for Windows only) from original installation DVD that I’ve bought many years ago and I was able to play this game under Linux! ๐Ÿ™‚

Posted in Linux | Tagged | Leave a comment

Copying data from Android mobile phone to PC with Linux

This time I’ve decided to make backup of data from my Android-based mobile phone and copy them to my Linux box. I wanted to do it without exposing things to a cloud (for sake of paranoid security). Data must have been copied directly to my PC:

  1. photos
  2. contact book – with ability to restore on another Android-based mobile
  3. SMS/MMS messages
  4. call registry
  5. calendar entries – with ability to restore on another Android-based mobile
  6. notes – with ability to restore on another Android-based mobile

In this case the version of Android was 5.1.1 (Samsung Galaxy Core Prime) and the PC was running under Linux Mint 19.3.

Ad.1. Photos

This was pretty simple. You can connect an Android-based mobile to a PC using a USB cable and modern Linux will handle it without any extra care so you can just copy JPEG files with photos. I’ve noticed 2 issues:
– When connecting to USB 3.0 it didn’t work – just try another USB socket in your computer.
– There are 2 modes Android-based mobiles can expose itself to a computer by USB interface, only one of them was correct (“multimedia device”).

Ad.2. Contact book

Step 1: On the mobile phone go to Contacts application, tap on “3 dots” symbol, then select “Settings”, then select “Contacts” and then “Import/Export contacts”. Then you select something like “Export to device memory”. The popup will appear showing you a path to a the file (VCF – vCard format) that will be created with all contacts’ data. Select OK to proceed.

Step 2: Connect the mobile to the PC by USB cable and copy the VCF file to your computer, just like with photos.

Ad.3. messages & Ad.4. call registry

Step 1: Install an application on the mobile phone: “SMS Backup & Restore” by SyncTech Pty Ltd. I used version 10.08.001.

Step 2: Open this application and select “Your phone” (PL: “Zapisz kopiฤ™ lokalnฤ…”) as backup target. Then return to the main screen and select creating a backup, then select both messages and calls. On this screen the application will show names of 2 XML files that will be created: one with messages and one with call registry. Then confirm creating a backup.

Step 3: Connect the mobile to the PC by USB cable, find these 2 files and copy them to your computer.

Ad.5. Calendar entries

Here I refer to calendar data maintained by the default Samsung calendar application (PL: “S Terminarz”, EN: “S Planner”) which is totally missing any import/export functionality.

Step 1: Install an application on the mobile phone: “Calendar Import – Export (ics)” by Sufficiently Secure.

Step 2: Open the application. On the first screen you need to select “a calendar” to work with. In my case the interesting calendar was not the one selected by default but the other one – its name consisted of my e-mail address that I used on this mobile to log into Google Play. After selecting the correct calendar the screen shows number of events present. As I had some calendar entries this was showing non-zero number – good sign! Then press “EXPORT” button on the bottom of the screen. The application will ask for the filename to be used – just give selfexplaining name without extension and press “OK”.

Step 3: Connect the mobile to the PC by USB cable and find the file. It will have ICS extension. Then just copy it to your computer.

You can use GNOME Evolution program under Linux to import ICS files and then view calendar entries.

BTW: This Android application “Calendar Import – Export (ics)” has the “import” function as well so it shouls be (in theory) possible to restore calendar entries on another Android mobile.

Ad.6. Notes

Here things are getting pretty complex – see this. Of course the built-in Samsung application for notes does not support any sharing or export functionality.
TODO. Do you know any hints? Please share…

Posted in hardware, Linux, security | Tagged | Leave a comment

My recommendations of programs for Linux

This is a list of my own recommendations of programs for Linux. I’ve used all of them, I was surprised by their quality and I wanted to list them here so I (and you) can easily find their names. I’m presenting them by categories. I will keep updating this page.

Screenshots tool

Use Flameshot. This is exactly how screenshot tool should look like! You should be able to install it on a Linux Mint or Ubuntu by this command:

sudo apt install flameshot

Web browser

Use The Brave Browser – no ads, no tracking, saving CPU, really nice! I even see much less ads (almost none?) on YouTube and other sites forcing to watch video ads before allowing them to see a video content.

This browser has built-in Torrent client:
Torrent client in Brave browser
Update (28.01.2021): this looks nice but I cannot recommend this feature any more as there is a drawback: a downloaded file is kept in browser’s memory. I was so dissappointed that Brave tab with Torrent client crashed exactly after downloading 2GB of Linux installation image and before having it written on a filesystem! After this I see this built-in client useless.

Installation instructions are here. The relevant part is:

curl -s https://brave-browser-apt-release.s3.brave.com/brave-core.asc | sudo apt-key --keyring /etc/apt/trusted.gpg.d/brave-browser-release.gpg add -
echo "deb [arch=amd64] https://brave-browser-apt-release.s3.brave.com/ stable main" | sudo tee /etc/apt/sources.list.d/brave-browser-release.list
sudo apt update
sudo apt install brave-browser

Diff tool

Use Meld. On Linux Mint or Ubuntu you will find it in the package manager under the name “meld”.

Games

Strategy game

Choose The Battle for Wesnoth! I was shocked that an open source video game can look that good! On Linux Mint or Ubuntu you will find it in the package manager under the name “wesnoth”.

Games for kids

If you have kids these games are something for them. All are open-source and can be installed from packages from standard repositories (Ubuntu, Linux Mint, etc.).

  1. Frogatto
  2. Frozen Bubble – has 2 players mode
  3. OpenTTD
  4. Extreme Tux Racer

Commercial games

I can recommend the Steam platform for Linux if you want to play some commercial video games. The nice thing is Linux-supporting games are marked (by the Steam icon…). I’ve successfully played on Linux following games from Steam:

  1. Grid Autosport – great car racing video game supporting split screen mode. Sad thing is the game is no longer on sale… How to play it now? :-\
  2. Another World (20th Anniversary Edition) – a remastered game from 1991!

CD/DVD burner

I recommend Brasero. Nowadays I burn DVDs very rarely. Thus I’m not good at this and I don’t remember extra hints when I need to burn a disk. Brasero is really simple and intuitive. At least when it comes to write ISO image on a DVD RW. On Linux Mint or Ubuntu you will find it in the package manager under the name “brasero”.

Astronomy

I definitely recommend Celestia – the real-time 3D visualization of space. It will really give you an impression how big the universe is!

Text editor for big files

My experience shows that the SciTE Text Editor is the best program to open and edit big text files. You can install it from the repostiory. The package name is “scite”.

Torrent

The Transmission is the Torrent client that is very simple and it never dissappointed me (as opposed to a Torrent client built in the Brave browser). This program is usually preinstalled on Linux Mint distributions. Anyway it should be available in Ubuntu repositories.

Posted in Linux | Leave a comment

Linux Mint 19.3 and HP Laser Jet 1018 printer

This is a really nice success story! I own the HP Laser Jet 1018 printer for more than 10 years now. It used to be a nightmare to make it running under Linux. But this time the setup worked like a charm!

Assuming you have a recent Linux Mint, like 19.3 and having the printer connected to your computer via USB cable, all you need to do to setup this printer is to follow instructions on the official HP web page: What is the HPLIP Binary Plug-In and How Do I Install It?. In short you just execute this command:

hp-setup

This will run a very nice GUI wizard which will guide you and will complete the configuration with couple of simple steps! Nothing needs to be installed manually by you. In my case the command hp-setup was just available in Linux Mint 19.3 after installation.

Well done Linux & HP!

Update

Ok, I was too much enthusiastic. On another computer with a really fresh installation of Linux Mint 19.3 it turned out that the command hp-setup shows an error message like this:

HP Linux Imaging and Printing System (ver. 3.17.10)
Printer/Fax Setup Utility ver. 9.0

Copyright (c) 2001-15 HP Development Company, LP
This software comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to distribute it
under certain conditions. See COPYING file for more details.

warning: GUI Modules PyQt4 and PyQt5 are not installed
error: hp-setup requires GUI support (try running with --qt3). Also, try using interactive (-i) mode.

Moreover it turned out running with –qt3 doesn’t work at all (how to report this?). And unfortunately using interactive (-i) mode didn’t give positive results.

Solution

In this case we need to install one thing:

sudo apt-get install python3-pyqt5

After this is successfully done execute the command hp-setup again. In my case GUI showed up:

Update: Linux Mint 20.1

On a fresh installation of Linux Mint 20.1 after plugging the printer (USB cable) one can see following system logs (using dmesg -w command):

[ 3665.125528] usb 1-2: new high-speed USB device number 2 using xhci_hcd
[ 3665.294232] usb 1-2: New USB device found, idVendor=03f0, idProduct=4117, bcdDevice= 1.00
[ 3665.294236] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3665.294239] usb 1-2: Product: HP LaserJet 1018
[ 3665.294241] usb 1-2: Manufacturer: Hewlett-Packard
[ 3665.294242] usb 1-2: SerialNumber: ...
[ 3674.709450] usblp 1-2:1.0: usblp0: USB Bidirectional printer dev 2 if 0 alt 0 proto 2 vid 0x03F0 pid 0x4117
[ 3674.709493] usbcore: registered new interface driver usblp
[ 3679.790584] usblp0: removed
[ 3679.808428] usblp 1-2:1.0: usblp0: USB Bidirectional printer dev 2 if 0 alt 0 proto 2 vid 0x03F0 pid 0x4117
[ 3780.692860] usblp0: removed

And then the desktop layer, Cinnamon in my case, shows a nice tooltip saying that the printer “HP-LaserJet-1018” was added. A printer icon appears in the system tray. However this is misleading – the printer is not yet ready. Trying to print something does nothing. I tried to follow the same procedure as for Linux Mint 19 but it failed on the phase “Driver Plug-in Installation” when using recommended mode (Download and install the plug-in from an HP authorized server). The error message printed on the console was:

error: /home/krzysiek/.hplip/hplip-3.20.11-plugin.run file does not match its checksum. File may have been corrupted or altered

Solution for Linux Mint 20.1

  1. First, install one missing dependency:
    sudo apt-get install python3-pyqt5
    
  2. Then make sure the printer is turned on and connected to your computer (USB). Then run the HP setup program:
    hp-setup
    

    On the console you should see output starting with:

    HP Linux Imaging and Printing System (ver. 3.20.11)
    Printer/Fax Setup Utility ver. 9.0
    
  3. It should detect the printer and show you “Driver Plug-in Installation” window. For the first time just click “Next” button to see how it goes. In my case on this step, after downloading the plugin, the hp-setup program claimed the file’s checksum is wrong. Take a look on the console (the terminal window where you executed “hp-setup” command) what is the exact name of the plugin file we need. In my case it was hplip-3.20.11-plugin.run. The file version matches the version of the hp-setup program.
  4. Open a web browser and go to https://www.openprinting.org/download/printdriver/auxfiles/HP/plugins/. On the page find and download 2 files: one file with exactly the same name, hplip-3.20.11-plugin.run, and another file with the name extended by “.asc” post-fix: hplip-3.20.11-plugin.run.asc. Save both files in the same directory.
  5. Go back to the HP setup wizard. Click “OK” button to close the dialog window saying “Please run ‘hp-plugin’ as normal user to install plug-ins”. And click “Next” button again on the window with text “Select From Discovered Devices”.
  6. You will again land on the window “Driver Plug-in Installation”. This time select second mode (Select and install an existing local copy of the plug-in file). Then click on a small folder icon on the right and select the plugin file (the one with “.run” extension, hplip-3.20.11-plugin.run in my case). Then click “Next” button.
  7. The program should show you “Driver Plug-in License Agreement” window. From now on the process is the same as in the procedure for the Linux Mint 19! ๐Ÿ™‚

After this the printer should make its characteristic noise. This means it’s OK.

PS. Many thanks to the Open Printing organization for providing HP printer plugin files!

Posted in hardware, Linux | Tagged , , , | Leave a comment

Out of memory when inserting lots of records with Hibernate

Issue

Recently my team was developing an application in Java that needed to insert a lots of records (around one million) into a relational database (AWS Aurora in MySQL mode). We used Spring Boot 2 together with a recent version of Spring Data JPA and Hibernate 5.3 underneath. Records were read from an input file (XML) and persisted in chunks of size up to 250. Each chunk was persisted in a separate database transaction and the whole process was handled by a single thread. It quickly turned out we were facing issue with lack of JVM heap memory.

We were surprised because we were convinced that Hibernate is clearing its cache at the end of a transaction (commit or rollback). A close examination using a memory profiler (jvisualvm) revealed that plenty of entity class objects were held by… Hibernate session manager (the session cache).

A close examination of Hibernate documentation section “Session batching” quickly explained what we have experienced and clarified there is no Hibernate bug involved.

Conclusion

Hibernate caches all newly inserted entities in the session-level cache and they are not removed from this cache when the transaction ends. So in context of batch processes one needs to subsequently create and close Hibernate sessions (this may be inefficient) for parts of the job or explicitly clear Hibernate session cache at some intervals. The second one is the solution recommended by the Hibernate documentation.

Solution

After persisting each chunk of records we have added flushing and clearing the Hibernate Session, like presented in the Hibernate documentation in the section “Batch inserts”. The issue with out of memory was gone.

Posted in database, Java | Tagged | Leave a comment