Category: Linux

Proxmox enable SOL access to ASRock Rack

I had some trouble enable the SOL (Serial-Over-LAN) access to my ASRock Rack motherboard. Here’s how I finally managed to get it working.

First start off by configuring “Serial Console Redirection” in the BIOS with the following settings (credit to pvalkone at GitHub):

/----------------------------------------------------+-------------------------\
|  SOL                                               |Emulation: ANSI:         |
|  Console Redirection Settings                      |Extended ASCII char      |
|                                                    |set. VT100: ASCII char   |
|  Terminal Type           [VT100]                   |set. VT100+: Extends     |
|  Bits per second         [115200]                  |VT100 to support color,  |
|  Data Bits               [8]                       |function keys, etc.      |
|  Parity                  [None]                    |VT-UTF8: Uses UTF8       |
|  Stop Bits               [1]                       |encoding to map Unicode  |
|  Flow Control            [None]                    |chars onto 1 or more     |
|  VT-UTF8 Combo Key Sup   [Enabled]                 |-------------------------|
|  Recorder Mode           [Disabled]                |<>: Select Screen        |
|  Resolution 100x31       [Disabled]                |^v: Select Item          |
|  Legacy OS Redirection   [80x24]                   |Enter: Select            |
|  Putty KeyPad            [VT100]                   |+/-: Change Option       |
|  Redirection After BIO   [Always Enable]           |F1: General Help         |
|                                                    |F7: Discard Changes      |
|                                                    |F9: Load UEFI Defaults   |
|                                                    |F10: Save and Exit       |
|                                                    |ESC: Exit                |
\----------------------------------------------------+-------------------------/

From your client (where you have installed ipmitools ):

➞ ipmitool -I lanplus -H <bmc-mgmt-ip> -U admin -e \& sol info 1
Password: 
Set in progress                 : set-complete
Enabled                         : true
Force Encryption                : false
Force Authentication            : false
Privilege Level                 : USER
Character Accumulate Level (ms) : 60
Character Send Threshold        : 96
Retry Count                     : 7
Retry Interval (ms)             : 500
Volatile Bit Rate (kbps)        : 115.2
Non-Volatile Bit Rate (kbps)    : 115.2
Payload Channel                 : 1 (0x01)
Payload Port                    : 623

Make sure that Volatile Bit Rate (kbps) and Non-Volatile Bit Rate (kbps) is set to 115.2 (115200).

Restart the server and login. Edit /etc/default/grub and change GRUB_CMDLINE_LINUX_DEFAULT, GRUB_TERMINAL and GRUB_SERIAL_COMMAND:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS1,115200n8 console=tty0"
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial --speed 115200 --unit=1 --word=8 --parity=no --stop=1"

After saving the file, do not forget to run update-grub.

Then we need to tell systemd to start serial ttyS1 on boot.

Optional, it can be worth changing the default serial-getty@.service to only accept 115200 as baud-rate, as according to Roger Irvin. Do this by coping the default service to /etc/systemd/system/.

cp /lib/systemd/system/serial-getty@.service /etc/systemd/system/

Then change the ExecStart line from:

ExecStart=-/sbin/agetty -o '-p -- \\u' --keep-baud 115200,38400,9600 %I $TERM

To:

ExecStart=-/sbin/agetty -o '-p -- \\u' 115200 %I $TERM'

Save the file, reload systemd daemon, start and enable ttyS1:

systemctl daemon-reload
systemctl start serial-getty@ttyS1
systemctl enable serial-getty@ttyS1

Reboot the server, and use ipmitool to get a SOL session:

ipmitool -I lanplus -H <bmc-mgmt-ip> -U admin -e \& sol activate

To exit the session type &. (no enter!) and the session is terminated.

OpenFaaS playground

I was playing around with OpenFaaS, and needed an, local, environment for it.

Install Ubuntu 20.04 Server in a virtual machine.

Install docker:

sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo usermod -a -G docker $USER

Install arkade and kubectl:

curl -sLS https://dl.get-arkade.dev | sudo sh
arkade get kubectl
echo "export PATH=\$PATH:\$HOME/.arkade/bin" >> ~/.bashrc
. ~/.bashrc

Download the latest version of minikube and start a new kubernetes cluster with docker as “backend”:

sudo dpkg -i minikube_latest_amd64.deb
minikube start --driver=docker

Deploy a private docker registry (domain docker-registry), which should be accessible from the host (ubuntu server), minikube (docker) and the namespace where OpenFaas function is being deployed:

arkade install docker-registry

In the output you’ll see the password for the admin user, we need it later on so make sure to save it. We also need to be able to access the registry “externally” (outside of the kubernetes cluster) with a self-signed certificate:

export REGISTRY_PASSWORD=<password from output above>
kubectl expose deploy/docker-registry --type=NodePort --name=docker-registry-external --port=5000
minikube addons enable ingress
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -out docker-registry-ingress-tls.crt \
    -keyout docker-registry-ingress-tls.key \
    -subj "/CN=docker-registry/O=docker-registry-ingress-tls"
kubectl create secret tls docker-registry-ingress-tls \
    --key docker-registry-ingress-tls.key \
    --cert docker-registry-ingress-tls.crt
cat <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-body-size: 2048m
    nginx.ingress.kubernetes.io/rewrite-target: /$1
  name: docker-registry-ingress
  namespace: default
spec:
  rules:
    - host: docker-registry
      http:
        paths:
          - path: /(.*)
            pathType: Prefix
            backend:
              service:
                name: docker-registry-external
                port:
                  number: 5000
  tls:
    - hosts:
      - docker-registry
      secretName: docker-registry-ingress-tls
EOF | kubectl apply -f -
sudo mkdir /usr/local/share/ca-certificates/docker-registry
sudo chmod 755 /usr/local/share/ca-certificates/docker-registry
sudo cp docker-registry-ingress-tls.crt /usr/local/share/ca-certificates/docker-registry
sudo chmod 644 /usr/local/share/ca-certificates/docker-registry/*
sudo update-ca-certificates
sudo mkdir -p /etc/docker/certs.d/docker-registry:443
sudo cp docker-registry-ingress-tls.crt /etc/docker/certs.d/docker-registry:443/ca.crt
scp -i $(minikube ssh-key) docker-registry-ingress-tls.crt docker@$(minikube ip):/home/docker
minikube ssh
sudo mkdir /usr/local/share/ca-certificates/docker-registry
sudo chmod 755 /usr/local/share/ca-certificates/docker-registry
sudo cp docker-registry-ingress-tls.crt /usr/local/share/ca-certificates/docker-registry
sudo chmod 644 /usr/local/share/ca-certificates/docker-registry/*
sudo update-ca-certificates
sudo mkdir -p /etc/docker/certs.d/docker-registry:443
sudo cp docker-registry-ingress-tls.crt /etc/docker/certs.d/docker-registry:443/ca.crt
sudo kill -SIGHUP $(pidof dockerd)
sudo apt update && sudo apt install -y vim-tiny
sudo vim.tiny /etc/hosts # add docker-registry after minikube
exit

Install and deploy OpenFaaS and the command line tool, and login to the OpenFaas gateway:

arkade install faas-cli
arkade install openfaas
kubectl port-forward -n openfaas svc/gateway 8080:8080 &
PASSWORD=$(kubectl get secret -n openfaas basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode; echo)
echo -n $PASSWORD | faas-cli login --username admin --password-stdin

The deployed functions need authentication for the private docker registry:

kubectl create secret -n openfaas-fn docker-registry docker-registry-credentials --docker-server=docker-registry:443 --docker-username=admin --docker-password=$REGISTRY_PASSWORD --docker-email=docker@example.com
kubectl edit serviceaccount default -n openfaas-fn

In the editor, add the following lines:

imagePullSecrets:
- name: docker-registry-credentials

Create a docker, client, configuration file with the basic authentication for the private docker registry:

mkdir ~/.docker/
cat > ~/.docker/config.json <<EOF
{
        "auths": {
                "docker-registry:443": {
                        "auth": "$(echo -n "admin:$REGISTRY_PASSWORD" | base64)"
                }
        }
}
EOF

That should be it.

Disconnecting from PPTP VPN – no network

I’m working with a client that has a PPTP VPN connection to get into their network. After disconnecting from the VPN, I would loose my connectivity to everything.
The routing table looked good, i.e. no default gateway or similar pointing to an incorrect gateway. Did some googling but couldn’t find any solution for it.

A workaround would be to disconnect and then connect to my LAN again. So after some more research I found out that you can create scripts in /etc/NetworkManager/dispatcher.d. These scripts will be feeded with two arguments: interface and state.

Create /etc/NetworkManager/dispatcher.d/80-ppp-vpn-down with the following contents:

#!/usr/bin/env bash

##
# The environment contains more information about the interface and the connection:
# DEVICE_IFACE - The interface name of the control interface of the device

main() {
        local interface="$1"
        local event="$2"

        if [[ "${interface}" =~ "ppp"* && "${event}" == "vpn-down" ]]; then
                local connection
                connection="$(nmcli -t -f NAME,DEVICE connection show --active | awk -F: '/'"${DEVICE_IFACE}"'/ {print $1}')" || return 1

                nmcli connection down id "${connection}"
                nmcli connection up id "${connection}"
        fi
}

main "$@"
exit $?

Don’t forget to make it executable:

chmod +x /etc/NetworkManager/dispatcher.d/80-ppp-vpn-down

Now when disconnecting from a PPTP connection, your connection used on the control interface will be toggled and you’ll have network connectivity again.

Ubuntu 18.04: Bad password, when trying to unlock LUKS encrypted device

I upgraded my two laptops from Ubuntu 17.10 to Ubuntu 18.04.

The first laptop (my backup) had two failed packages (shim-signed and grub-efi-amd64-signed). Since I don’t have secured boot enabled, I just removed these packages and everything was fine.

The second laptop (my primary) had one failed package (ca-certificates), which seemed a little less problematic then the other two packages, so I rebooted.
When asked for the passphrase to unlock my encrypted root device, I just got “bad password or options?”. Enter panic mode.

A google search showed a lot of `cryptsetup` problems in 18.04. However none seems to be the same problem as I had.

I tried booting a live usb-stick to unlock and mount the device there, but once again: “bad password”.

Gave it up for a couple of days when I finally got around to try with a 16.04 live usb-stick, same problem though.

For some reason, I tried entering the password with the characters that it should’ve been if I was using US layout.
Say my password contained = (swedish layout), I replaced those characters with ) (the character that the same key in US layout will give) and it actually worked! Note that I always changed to Swedish layout on the live system.

So it seemed like the upgrade actually changed my passphrase. There has probably been a bug that cryptsetup passphrases has been entered with US layout, and a the passphrase prompt it has always been US layout, and now it is fixed, so the passphrase prompt is with what the layout used in the system. Hence the old passphrase is not working. Re-reading this answer (and the comments that has been added since I read it the first time) actually makes a lot more sense now.

After successfully unlocking my root device, I added a “new” passphrase and removed the old one (actually a bunch of reboots inbetween to make sure the passphrases worked):

sudo cryptsetup luksAddKey /dev/sdXY
sudo cryptsetup luksRemoveKey /dev/sdXY

DisplayLink and Ubuntu 17.04 (intel)

I had a problem with DisplayLink drivers on my Dell Precision 5510 (XPS 15) after upgrading to Ubuntu 17.04.

The monitors connected to the DisplayLink device was enabled and identified, but the screens stayed all black.

Of course, there’s a solution. I found this evdi issue, containing a solution.

Create the file (and directory) /etc/X11/xorg.conf.d/20-intel.conf, with the content:

Section "Device"
  Identifier "Intel Graphics"
  Driver "Intel"
  Option "AccelMethod" "sna"
  Option "TearFree" "true"
  Option "TripleBuffer" "true"
  Option "MigrationHeuristic" "greedy"
  Option "Tiling" "true"
  Option "Pageflip" "true"
  Option "ExaNoComposite" "false"
  Option "Tiling" "true"
  Option "Pageflip" "true"
EndSection

Credit goes to github/ajbogh.