Sunday 15 September 2019

Ubuntu 18.04 no DNS after VPN

Symptoms

On Ubuntu 18.04, once you connect to VPN (OpenVPN, L2TP or other point-to-point protocols), your system will use the remote peer's DNS server, unless explicitly configured not to use peer DNS. Once the VPN session is closed, local name resolution does not work.

Root cause and quickfix

Turning off and then back on the physical network interface (WiFi or wired) fixed the issue. On this version of Ubuntu, systemd-resolve is responsible for managing changes to, and configuration of domain name resolution. Running systemd-resolve --status correctly shows the original nameserver that was assigned when the connection via the physical interface was established, however this nameserver is not marked as the default and in fact there is no default nameserver once the VPN session is closed.

A quick command like the following recovers the issue: systemd-resolve --set-dns=192.168.0.1 --interface=wlp4s0 but does not resolve the root cause. Further analysis has shown that the error is caused by a file permission issue due to lacking integration of systemd, network manager and ppp. While tracking down such issues can be fun, this case was more annoying than useful

Solution candidate

As in most of the cases, a bit of googling unveils that this problem also affects others. There is a bug report in launchpad, which is marked as resolved with a fix. Unfortunately, that was not applied to 18.04 although it is a long term release. Nevertheless, one can inspect the patch and find that adding a few lines to /etc/ppp/ip-up.d/0000usepeerdns should fix the issue. The point of the patch is to prevent the misbehaving DNS configuration scripts to run when network-manager initiated VPN connections are terminated.

Applying patches directly to the local file system is not my preferred option - I would rather repackage the affected ppp deb or use dpkg-divert to apply local changes in a way that plays nicely together with the packaging system and keeps my system easily manageable in the long run. However, for a quick verification, I have applied the required modification, and found it would not work per se.

Applied solution

Switching to plan B. Upon inspecting /etc/ppp/ip-up.d/0000usepeerdns once can clearly see that if resolvconf - the traditional way how Ubuntu and most Linux distributions would manage name resolution in previous releases - is installed, the problematic part of the script is not executed. I settled with this simpler and cleaner solution, installed the resolvconf package and with that the issue was resolved.

Monday 26 August 2019

Ubuntu 18.04 on ThinkPad P50 (Part 4)

Conflict between libcurl3 and libcurl4

Ubuntu 18.04 includes libcurl4 by default which in fact is required by the packaged version of curl. Some legacy applications depend on libcurl3, which but libcurl3 conflicts with libcurl4. The steps below demonstrate how to alter the control file of such legacy packages to allow libcurl4 instead of libcurl3 and then how to repackage and install these packages.

People following these steps should test the updated legacy applications and confirm they run properly, or revert if they face any issue they cannot resolve.


tb@castor:~$ cd /tmp
tb@castor:/tmp$ mkdir dpkg
tb@castor:/tmp$ cd dpkg/
tb@castor:/tmp/dpkg$ apt download xxxsam osprules 
Get:1 http://ocdc.hursley.xxx.com/ocdc bionic-safe/XXX amd64 xxxsam amd64 3.06-4-ocdc6 [195 kB]
Get:2 http://ocdc.hursley.xxx.com/ocdc bionic-safe/XXX amd64 osprules amd64 1.00.1-2.precise-ocdc2 [34,7 kB]
Fetched 230 kB in 0s (708 kB/s)    
tb@castor:/tmp/dpkg$ ll
total 228
drwxr-xr-x  2 tb   tb       80 aug   26 17:54 ./
drwxrwxrwt 26 root root    680 aug   26 17:54 ../
-rw-r--r--  1 tb   tb   194938 márc  20  2015 xxxsam_3.06-4-ocdc6_amd64.deb
-rw-r--r--  1 tb   tb    34718 júl   13  2012 osprules_1.00.1-2.precise-ocdc2_amd64.deb
tb@castor:/tmp/dpkg$ fakeroot
root@castor:/tmp/dpkg# dpkg-deb -R xxxsam_3.06-4-ocdc6_amd64.deb xxxsam
root@castor:/tmp/dpkg# dpkg-deb -R osprules_1.00.1-2.precise-ocdc2_amd64.deb ospfules
root@castor:/tmp/dpkg# sed -i 's/libcurl3 (>= 7.16.2-1)/libcurl3 (>= 7.16.2-1) | libcurl4/' xxxsam/DEBIAN/control
root@castor:/tmp/dpkg# sed -i 's/libcurl3 (>= 7.16.2-1)/libcurl3 (>= 7.16.2-1) | libcurl4/' ospfules/DEBIAN/control
root@castor:/tmp/dpkg# sed -i 's/\(^Version:.*$\)/\1+dep1/' ospfules/DEBIAN/control
root@castor:/tmp/dpkg# sed -i 's/\(^Version:.*$\)/\1+dep1/' xxxsam/DEBIAN/control
root@castor:/tmp/dpkg# dpkg-deb -b xxxsam xxxsam_3.06-4-ocdc6+dep1_amd64.deb 
dpkg-deb: warning: conffile name '/etc/init.d/xxx-isambios' is duplicated
dpkg-deb: warning: conffile name '/etc/xdg/autostart/xxx-asset-management.desktop' is duplicated
dpkg-deb: warning: ignoring 2 warnings about the control file(s)
dpkg-deb: building package 'xxxsam' in 'xxxsam_3.06-4-ocdc6+dep1_amd64.deb'.
root@castor:/tmp/dpkg# dpkg -b ospfules osprules_1.00.1-2.precise-ocdc2+dep1_amd64.deb 
dpkg-deb: building package 'osprules' in 'osprules_1.00.1-2.precise-ocdc2+dep1_amd64.deb'.
root@castor:/tmp/dpkg# ll
total 452
drwxr-xr-x  4 root root    160 aug   26 18:06 ./
drwxrwxrwt 26 root root    680 aug   26 18:06 ../
drwxr-xr-x  7 root root    140 aug   26 18:00 xxxsam/
-rw-r--r--  1 root root 194938 márc  20  2015 xxxsam_3.06-4-ocdc6_amd64.deb
-rw-r--r--  1 root root 195024 aug   26 18:04 xxxsam_3.06-4-ocdc6+dep1_amd64.deb
drwxr-xr-x  4 root root     80 aug   26 18:00 ospfules/
-rw-r--r--  1 root root  34718 júl   13  2012 osprules_1.00.1-2.precise-ocdc2_amd64.deb
-rw-r--r--  1 root root  29128 aug   26 18:06 osprules_1.00.1-2.precise-ocdc2+dep1_amd64.deb
root@castor:/tmp/dpkg# exit
exit
tb@castor:/tmp/dpkg$ sudo apt install -f ./xxxsam_3.06-4-ocdc6+dep1_amd64.deb 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'xxxsam' instead of './xxxsam_3.06-4-ocdc6+dep1_amd64.deb'
The following packages will be upgraded:
  xxxsam
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/195 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 /tmp/dpkg/xxxsam_3.06-4-ocdc6+dep1_amd64.deb xxxsam amd64 3.06-4-ocdc6+dep1 [195 kB]
(Reading database ... 369572 files and directories currently installed.)
Preparing to unpack .../xxxsam_3.06-4-ocdc6+dep1_amd64.deb ...
Unpacking xxxsam (3.06-4-ocdc6+dep1) over (3.06-4-ocdc6) ...
Setting up xxxsam (3.06-4-ocdc6+dep1) ...
.
Processing triggers for systemd (237-3ubuntu10.25) ...
Processing triggers for ureadahead (0.100.0-21) ...
tb@castor:/tmp/dpkg$ sudo apt install -f ./osprules_1.00.1-2.precise-ocdc2+dep1_amd64.deb 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'osprules' instead of './osprules_1.00.1-2.precise-ocdc2+dep1_amd64.deb'
The following packages will be upgraded:
  osprules
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/29,1 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 /tmp/dpkg/osprules_1.00.1-2.precise-ocdc2+dep1_amd64.deb osprules amd64 1.00.1-2.precise-ocdc2+dep1 [29,1 kB]
(Reading database ... 369572 files and directories currently installed.)
Preparing to unpack .../osprules_1.00.1-2.precise-ocdc2+dep1_amd64.deb ...
Unpacking osprules (1.00.1-2.precise-ocdc2+dep1) over (1.00.1-2.precise-ocdc2) ...
Setting up osprules (1.00.1-2.precise-ocdc2+dep1) ...
Creating symbolic links ...
tb@castor:/tmp/dpkg$ 

Thursday 22 August 2019

Ubuntu 18.04 on ThinkPad P50 (Part 3)

Repackaging Lotus Notes for 18.04

Some of our corporate Domino applications still do not work properly when accessed from a web browser, so there was a need to install the Notes client on Ubuntu 18.04. This involved updating deprecated dependencies, rebuilding the deb package and publishing it to the local package repository.


tb@castor:~$ cd /tmp
tb@castor:/tmp$ mkdir notes
tb@castor:/tmp$ apt-get download ibm-notes
Get:1 http://ocdc.hursley.xyz.com/ocdc bionic-safe/XYZ i386 ibm-notes i386 9.0.1-20131022.1138+ocdc7 [358 MB]
Fetched 358 MB in 20min 8s (296 kB/s)                                          
tb@castor:/tmp/notes$ fakeroot
root@castor:/tmp/notes# dpkg -x ../ibm-notes*.deb .
 ...
root@castor:/tmp/notes# dpkg -e ../ibm-notes*.deb ./DEBIAN
root@castor:/tmp/notes# sed -i 's/libpng12-0/libpng16-16/g' DEBIAN/control 
root@castor:/tmp/notes# sed -i 's/+ocdc7/+ocdc8/g' DEBIAN/control         
root@castor:/tmp/notes# gunzip usr/share/doc/ibm-notes/changelog.Debian.gz 
root@castor:/tmp/notes# nano usr/share/doc/ibm-notes/changelog.Debian # append to changelog

root@castor:/tmp/notes# cd ..
root@castor:/tmp# dpkg -b /tmp/notes ibm-notes_9.0.1-20131022.1138+ocdc8_i386.deb 
dpkg-deb: building package 'ibm-notes' in 'ibm-notes_9.0.1-20131022.1138+ocdc8_i386.deb'.

root@castor:/tmp# exit
exit
tb@castor:/tmp/notes$ cd /usr/local/lib/apt/amd64/
tb@castor:/usr/local/lib/apt/amd64$ sudo cp /tmp/ibm-notes_9.0.1-20131022.1138+ocdc8_i386.deb .
tb@castor:/usr/local/lib/apt/amd64$ apt-ftparchive packages . | sudo tee Packages
 ...
tb@castor:/usr/local/lib/apt/amd64$ apt-ftparchive release . | sudo tee Release
Date: Wed, 07 Aug 2019 16:16:51 +0000
MD5Sum:
 7339b2b2be699eb389665b64b062c1fe             6319 Packages
 62234bf67aa71907789bcb7df1f05e97              444 Release
SHA1:
 7df336123833077c551c0285782d10aaf9cca53f             6319 Packages
 6e0d07f80c9062661e481fa159265b9ccbce09ff              444 Release
SHA256:
 2556cbebb467e8ec5b48733ac21fcb4b124ec252ea70cdf22cf9994e1e5d9849             6319 Packages
 77264df13b76bfc355aa3a911e10a4983d8737914fb1904d9a255d9c5a58d8d2              444 Release
SHA512:
 a1808ee41b7131d1fdb93ba32c5d04a496fafb8223d7b9e3ad3e00e0bfbfd138d75743a206f9beea5f724f5c63e4304bfff277a74e1087fa3047294b9b8b6459             6319 Packages
 5498ce04b5d3562d2cd48e4105eea0135059d9ea8a6ad61f57cb1e78f49841df629a9ab7a2cd2403f247f1d29a36ad5c04c21f69c6c00a10a09b7ee69e781e7d              444 Release
tb@castor:/usr/local/lib/apt/amd64$ 

First run and the missing bit

Notes silently failed and terminated after splash screen. No error logged, no trace of any missing libraries, no indication of the root cause. I had to find out the hard way, that the EULA acceptance prompt that was triggered on the first run was implemented in a way that relied on running scripts from /tmp. Which, in my case, was mounted with nosuid,nodev,noexec. I would really have appreciated at least a line of error message.


mount | grep /tmp
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,noatime)
sudo mount -o exec,remount /tmp
/opt/ibm/notes/framework/../notes0
sudo mount -o noexec,remount /tmp # can be remounted with noexec after first run and accepting license

The final step

Having fixed the EULA related issue above, I experienced that Notes would enter an infinite loop after successful login. I have just moved over the whole ~/lotus directory from my old thinkpad, and had to make changes to notes.ini to reflect my updated linux userid and home path. Once done, Notes started smoothly and did not experience any issue since

Wednesday 3 July 2019

Ubuntu 18.04 on ThinkPad P50 (Part 2)

Corporate open client repository

One of the first steps was to add Open Client Debian Community (OCDC) package repository. This is a package repository which includes mandatory and optional packages provided by the corporation (well, maintained by our internal Linux community), including endpoint management tools that enable registration and compliance checks. Expect more on this in future posts.


tb@castor:~$ sudo apt-get install libfrontier-rpc-perl libjson-perl libgtk2-perl
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libcommon-sense-perl libjson-xs-perl libpango-perl libtypes-serialiser-perl
Suggested packages:
  libgtk2-perl-doc
The following NEW packages will be installed:
  libcommon-sense-perl libfrontier-rpc-perl libgtk2-perl libjson-perl
  libjson-xs-perl libpango-perl libtypes-serialiser-perl
0 upgraded, 7 newly installed, 0 to remove and 26 not upgraded.
Need to get 923 kB of archives.
After this operation, 4.237 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libcommon-sense-perl amd64 3.74-2build2 [20,3 kB]
Get:2 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libfrontier-rpc-perl all 0.07b4-6 [35,5 kB]
Get:3 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libpango-perl amd64 1.227-2build1 [157 kB]
Get:4 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libgtk2-perl amd64 2:1.24992-1build1 [544 kB]
Get:5 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libjson-perl all 2.97001-1 [73,3 kB]
Get:6 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libtypes-serialiser-perl all 1.0-1 [12,1 kB]
Get:7 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libjson-xs-perl amd64 3.040-1 [81,2 kB]
Fetched 923 kB in 1s (632 kB/s)           
Selecting previously unselected package libcommon-sense-perl.
(Reading database ... 161358 files and directories currently installed.)
Preparing to unpack .../0-libcommon-sense-perl_3.74-2build2_amd64.deb ...
Unpacking libcommon-sense-perl (3.74-2build2) ...
Selecting previously unselected package libfrontier-rpc-perl.
Preparing to unpack .../1-libfrontier-rpc-perl_0.07b4-6_all.deb ...
Unpacking libfrontier-rpc-perl (0.07b4-6) ...
Selecting previously unselected package libpango-perl.
Preparing to unpack .../2-libpango-perl_1.227-2build1_amd64.deb ...
Unpacking libpango-perl (1.227-2build1) ...
Selecting previously unselected package libgtk2-perl.
Preparing to unpack .../3-libgtk2-perl_2%3a1.24992-1build1_amd64.deb ...
Unpacking libgtk2-perl (2:1.24992-1build1) ...
Selecting previously unselected package libjson-perl.
Preparing to unpack .../4-libjson-perl_2.97001-1_all.deb ...
Unpacking libjson-perl (2.97001-1) ...
Selecting previously unselected package libtypes-serialiser-perl.
Preparing to unpack .../5-libtypes-serialiser-perl_1.0-1_all.deb ...
Unpacking libtypes-serialiser-perl (1.0-1) ...
Selecting previously unselected package libjson-xs-perl.
Preparing to unpack .../6-libjson-xs-perl_3.040-1_amd64.deb ...
Unpacking libjson-xs-perl (3.040-1) ...
Setting up libcommon-sense-perl (3.74-2build2) ...
Setting up libtypes-serialiser-perl (1.0-1) ...
Setting up libfrontier-rpc-perl (0.07b4-6) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up libjson-perl (2.97001-1) ...
Setting up libpango-perl (1.227-2build1) ...
Setting up libgtk2-perl (2:1.24992-1build1) ...
Setting up libjson-xs-perl (3.040-1) ...
tb@castor:~$ cd /tmp
tb@castor:/tmp$ mkdir ocdc
tb@castor:/tmp$ cd ocdc/
tb@castor:/tmp/ocdc$ wget http://ocdc.xxxxxxx.yyy.com/ocdc/{yyy-ca-certificates,ocdc-archive-keyring,ocdc-repository}.deb
tb@castor:/tmp/ocdc$ sudo dpkg -i *.deb

Minor adjustments to outdated package dependencies

Some of the community maintained corporate packages include dangling references to packages which are not available any more on Ubuntu 18.04. Fixing the package itself is the first step. The command listing below demonstrated the process on a prominent example: I lifted dependency on consolekit and libpam-ck-connector, which are not available any more, then rebuilt the package.


tb@castor:~$ cd /tmp/ocdc/
tb@castor:/tmp/ocdc$ apt-get download acme-security-compliance 
tb@castor:/tmp/ocdc$ dpkg-deb -x acme-security-compliance_5.0.4.1-12_all.deb acme-security-compliance
tb@castor:/tmp/ocdc$ dpkg-deb --control acme-security-compliance_5.0.4.1-12_all.deb acme-security-compliance/DEBIAN 
tb@castor:/tmp/ocdc$ cat acme-security-compliance/DEBIAN/control 
Package: acme-security-compliance
Version: 5.0.4.1-12
Architecture: all
Maintainer: OCDC Core Team <ocdc-approvers@ocdc.hursley.acme.com>
Installed-Size: 356
Depends: libpam-runtime, python, perl, libpam-passwdqc, ecryptfs-utils, acme-security-rules, gconf2, procps, sysv-rc, gnome-keyring, libpam-gnome-keyring, consolekit, libpam-ck-connector
Section: ACME/utils
Priority: extra
Description: Implementation of ACME security guidelines
 The ACME Network and Computing Security Services (NCSS) group has defined
 a set of mandatory security guidelines to protect ACME's information assets
 on network and computing environments within ACME.  This is referred to as the
 ITCS104 Security Standards for Providers of Network and Computing Services.
 In addition to ITCS104, there is a commercial Information Security
 Controls Document referred to as GSD331 which recommends
 additional guidelines for Strategic Outsourcing commercial customers.
 For additional information on these guidelines, visit the Network and
 Computing Security Services (NCSS) webpage at http://w3.security.acme.com
 The Client for e-business implements all of the guidelines from ITCS104 and
 the most commonly used elements of GSD331.  These settings are applied to
 the requisite files by installing patched versions.
tb@castor:/tmp/ocdc$ nano acme-security-compliance/DEBIAN/control 
tb@castor:/tmp/ocdc$ cat acme-security-compliance/DEBIAN/control 
Package: acme-security-compliance
Version: 5.0.4.1-12+dep1
Architecture: all
Maintainer: OCDC Core Team <ocdc-approvers@ocdc.hursley.acme.com>
Installed-Size: 356
Depends: libpam-runtime, python, perl, libpam-passwdqc, ecryptfs-utils, acme-security-rules, gconf2, procps, gnome-keyring, libpam-gnome-keyring
Section: ACME/utils
Priority: extra
Description: Implementation of ACME security guidelines
 The ACME Network and Computing Security Services (NCSS) group has defined
 a set of mandatory security guidelines to protect ACME's information assets
 on network and computing environments within ACME.  This is referred to as the
 ITCS104 Security Standards for Providers of Network and Computing Services.
 In addition to ITCS104, there is a commercial Information Security
 Controls Document referred to as GSD331 which recommends
 additional guidelines for Strategic Outsourcing commercial customers.
 For additional information on these guidelines, visit the Network and
 Computing Security Services (NCSS) webpage at http://w3.security.acme.com
 The Client for e-business implements all of the guidelines from ITCS104 and
 the most commonly used elements of GSD331.  These settings are applied to
 the requisite files by installing patched versions.
tb@castor:/tmp/ocdc$ gunzip acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian.gz
tb@castor:/tmp/ocdc$ nano acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian
tb@castor:/tmp/ocdc$ head acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian
acme-security-compliance (5.0.4.1-12+dep1) experimental; urgency=low

  * Don't replace the gdm-autologin file for pam. Autologin settings will
    be enforced by WST/TEM, and replacing it can break gdm.

 -- Tibor Bősze <tibor.boesze@hu.acme.com>  Fri, 03 May 2019 14:23:00 +0200

acme-security-compliance (5.0.4.1-12) experimental; urgency=low

  * Don't replace the gdm-autologin file for pam. Autologin settings will
    be enforced by WST/TEM, and replacing it can break gdm.
tb@castor:/tmp/ocdc$ gzip acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian
tb@castor:/tmp/ocdc$
tb@castor:/tmp/ocdc$ dpkg -b acme-security-compliance acme-security-compliance_5.0.4.1-12+dep1_all.deb
dpkg-deb: building package 'acme-security-compliance' in 'acme-security-compliance_5.0.4.1-12+dep1_all.deb'.

I added the +dep1 suffix to the package rather than increasing the package version. This is meant to indicate a local diversion and does not interfere with updates. Once can pin/hold a given version via apt.

Setting up a local apt repository

The next step was to create a local package repository which would contain all the locally brewed packages in the future. The main advantage is that I can still benefit from all the advantages apt offers compared to bare-foot dpkg. At some point, these can be shared or pushed to upstream after some review and approval process.


tb@castor:/tmp/ocdc$ cd /usr/local/lib
tb@castor:/usr/local/lib$ sudo mkdir apt
tb@castor:/usr/local/lib/apt$ sudo mkdir amd64
tb@castor:/usr/local/lib/apt$ sudo cp /tmp/ocdc/*dep1*.deb amd64/
tb@castor:/usr/local/lib/apt$ cd amd64
tb@castor:/usr/local/lib/apt/amd64$ apt-ftparchive packages . | sudo tee Packages
Package: acme-security-compliance
Architecture: all
Version: 5.0.4.1-12+dep1
Priority: extra
Section: ACME/utils
Maintainer: OCDC Core Team <ocdc-approvers@ocdc.hursley.acme.com>
Installed-Size: 356
Depends: libpam-runtime, python, perl, libpam-passwdqc, ecryptfs-utils, acme-security-rules, gconf2, procps, gnome-keyring, libpam-gnome-keyring
Filename: ./acme-security-compliance_5.0.4.1-12+dep1_all.deb
Size: 38272
MD5sum: c7103b33f23a302de24fa270f50f8532
SHA1: d41315e1faad5e39d33bfd8a86cd3ae54beef3be
SHA256: 643b621bf832e0438a4e18b3725c60a33c9164c1f15d7f29e7c21652944c312e
SHA512: 920cb177fde140a4f08262039015f0f67aaf785e177c2a31d3c22a5d6dd5e3b3636653330beac7ef1c7b17675b1d2a78226ff07a3d6eea1e4922f2430e3ce7e9
Description: Implementation of ACME security guidelines
 The ACME Network and Computing Security Services (NCSS) group has defined
 a set of mandatory security guidelines to protect ACME's information assets
 on network and computing environments within ACME.  This is referred to as the
 ITCS104 Security Standards for Providers of Network and Computing Services.
 In addition to ITCS104, there is a commercial Information Security
 Controls Document referred to as GSD331 which recommends
 additional guidelines for Strategic Outsourcing commercial customers.
 For additional information on these guidelines, visit the Network and
 Computing Security Services (NCSS) webpage at http://w3.security.acme.com
 The Client for e-business implements all of the guidelines from ITCS104 and
 the most commonly used elements of GSD331.  These settings are applied to
 the requisite files by installing patched versions.

tb@castor:/usr/local/lib/apt/amd64$ cd ..
tb@castor:/usr/local/lib/apt$ apt-ftparchive release . | sudo tee Release
Date: Fri, 03 May 2019 13:32:52 +0000
MD5Sum:
 46a0d423952446e8b31e4c8b2f6db4fa             1252 Release
 f1001d18fbe7ab32cab6178efff454a8             1619 amd64/Packages
SHA1:
 4faf1d629ba57672840f742c30d9f3520c94ae54             1252 Release
 cb820ae5264f24af1d3e1b1c1c0ac8d20e134224             1619 amd64/Packages
SHA256:
 5ef09f42467d1c89209f83fbeae48a722126e14519df72eca6d0b1c899493221             1252 Release
 612cb061581b2e1edfa480dd3569e6eddcbb85d30bb49170394d21d21c789fef             1619 amd64/Packages
SHA512:
 eac63ffee19c0a2e34556b209c1907e86b776515571291eb9b8e7fe435627815e6324e53fa8c1a2f044f0815546e5d3ae2e12ac8900ca89534c7bef1c006bfcd             1252 Release
 302fabb5e99825dc0951a9a3bbeb22be711058fe5a63a19e50b624e466608152bdff2694a398280fbc5101fc75a2d64fdd8eef712f9dc6e7d1a807aa17809d75             1619 amd64/Packages
tb@castor:/usr/local/lib/apt$ find .
.
./Release
./amd64
./amd64/acme-security-compliance_5.0.4.1-12+dep1_all.deb
./amd64/Packages
tb@castor:/usr/local/lib/apt$ sudo nano /etc/apt/sources.list
tb@castor:/usr/local/lib/apt$ tail /etc/apt/sources.list
deb http://security.ubuntu.com/ubuntu bionic-security main restricted
# deb-src http://security.ubuntu.com/ubuntu bionic-security main restricted
deb http://security.ubuntu.com/ubuntu bionic-security universe
# deb-src http://security.ubuntu.com/ubuntu bionic-security universe
deb http://security.ubuntu.com/ubuntu bionic-security multiverse
# deb-src http://security.ubuntu.com/ubuntu bionic-security multiverse

## local repo
deb [trusted=yes] file:/usr/local/lib/apt/amd64 ./

tb@castor:/usr/local/lib/apt/amd64$ sudo aptitude
Performing actions...

...
 
Press Return to continue, 'q' followed by Return to quit.

tb@castor:/usr/local/lib/apt/amd64$

Removing some snaps

Snappy is a software deployment platform developed by Canonical, introducing the snap package format, which is intended to be less dependent on the version of operating system and libraries. Unlike rpm or deb but very much like FlatPack and AppImage, Snappy intents to offer a modern application delivery model shipping self contained software packages which enable developers to publish and update their applications with less distribution specific overhead, and allow end users to receive the most up-to-date version of their application in a smooth manner.

Snappy has received much criticism, and while I believe there are valid use cases for deploying applications via snaps, I dislike the idea that Ubuntu 18.04 by default ships some core application - such as system-monitor - as snap. I prefer to keep the core of my environment using good-old debs. I decided to only deploy snaps as exception, if there is a strong compelling reason to do so. Therefore, I removed some pre-installed snaps and replaced them with their deb counterparts.


tb@castor:/tmp/ocdc$ snap list
Name                  Version         Rev   Tracking  Publisher   Notes
core                  16-2.37.1       6350  stable    canonical✓  core
gnome-3-26-1604       3.26.0          74    stable/…  canonical✓  -
gnome-calculator      3.30.1          260   stable/…  canonical✓  -
gnome-characters      3.30.0          139   stable/…  canonical✓  -
gnome-logs            3.30.0          45    stable/…  canonical✓  -
gnome-system-monitor  3.30.0          57    stable/…  canonical✓  -
gtk-common-themes     0.1-4-g88bc1b2  818   stable/…  canonical✓  - 
tb@castor:/tmp/ocdc$ sudo snap remove gnome-calculator gnome-characters gnome-logs gnome-system-monitor 
2019-05-03T13:31:49+02:00 INFO Waiting for conflicting change in progress...
2019-05-03T13:31:52+02:00 INFO Waiting for conflicting change in progress...
2019-05-03T13:31:58+02:00 INFO Waiting for conflicting change in progress...
2019-05-03T13:32:09+02:00 INFO Waiting for conflicting change in progress...
gnome-calculator removed
gnome-characters removed
gnome-logs removed
gnome-system-monitor removed
tb@castor:/tmp/ocdc$ sudo apt-get install gnome-calculator gnome-characters gnome-logs gnome-system-monitor 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libatkmm-1.6-1v5 libcairomm-1.0-1v5 libglibmm-2.4-1v5 libgtkmm-3.0-1v5
  libpangomm-1.4-1v5
The following NEW packages will be installed:
  gnome-calculator gnome-characters gnome-logs gnome-system-monitor
  libatkmm-1.6-1v5 libcairomm-1.0-1v5 libglibmm-2.4-1v5 libgtkmm-3.0-1v5
  libpangomm-1.4-1v5
0 upgraded, 9 newly installed, 0 to remove and 26 not upgraded.
Need to get 2.585 kB of archives.
After this operation, 15,5 MB of additional disk space will be used.
Do you want to continue? [Y/n] 

To be fair, let me restate that I do see valid use cases for deploying snaps. I deployed slack via Snappy. It is closed source, third party app and I prefer to use a more recent version than what a 5-years LTS will realistically provide, and run it isolated in it's own sandbox.

Tuesday 25 June 2019

Ubuntu 18.04 on ThinkPad P50 (Part 1)

I am moving to a ThinkPad P50 from my W530 and decided to document the steps of preparing a 18.04 box which complies with our corporate policies. The first step is to enter firmware setup and go through the setting, then boot into a 18.04 LiveUSB and erase the preinstalled closed source OS. What follows is the custom LUKS/LVM disk setup done before running the installer, then running the installer itself, finally, a few extra steps which in themselves are simple but will cause head-scratching if skipped...

LUKS/LVM setup from the terminal

The terminal history should be self-explanatory for those who have followed up earlier posts, so rather than explaining each and every command in detail, I decided to just dump the exact commands and their output that was used to prepare the disk before running the installer.


To run a command as administrator (user "root"), use "sudo ".
See "man sudo_root" for details.

ubuntu@ubuntu:~$ sudo gdisk /dev/sda
GPT fdisk (gdisk) version 1.0.3

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sda: 1000215216 sectors, 476.9 GiB
Model: SAMSUNG MZNLN512
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 126AD2BA-4B0A-45B8-97D0-845A43DDD3C7
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1000215182
Partitions will be aligned on 2048-sector boundaries
Total free space is 2669 sectors (1.3 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          264191   128.0 MiB   EF00  EFI system partition
   2          264192         2099199   896.0 MiB   8300  BOOT
   3         2099200      1000214527   475.9 GiB   8300  SYSTEM

Command (? for help): q
ubuntu@ubuntu:~$ cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1      1466539 iterations per second for 256-bit key
PBKDF2-sha256    1688528 iterations per second for 256-bit key
PBKDF2-sha512    1322290 iterations per second for 256-bit key
PBKDF2-ripemd160 1052787 iterations per second for 256-bit key
PBKDF2-whirlpool  771011 iterations per second for 256-bit key
argon2i       6 iterations, 1048576 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
argon2id      6 iterations, 1048576 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
#     Algorithm | Key |  Encryption |  Decryption
        aes-cbc   128b  1033.7 MiB/s  3213.1 MiB/s
    serpent-cbc   128b    86.0 MiB/s   660.5 MiB/s
    twofish-cbc   128b   191.6 MiB/s   359.0 MiB/s
        aes-cbc   256b   801.6 MiB/s  2593.6 MiB/s
    serpent-cbc   256b    87.3 MiB/s   659.5 MiB/s
    twofish-cbc   256b   193.8 MiB/s   359.4 MiB/s
        aes-xts   256b  1959.5 MiB/s  1959.9 MiB/s
    serpent-xts   256b   635.3 MiB/s   651.2 MiB/s
    twofish-xts   256b   352.8 MiB/s   354.6 MiB/s
        aes-xts   512b  1819.2 MiB/s  1823.0 MiB/s
    serpent-xts   512b   638.8 MiB/s   650.9 MiB/s
    twofish-xts   512b   353.7 MiB/s   355.1 MiB/s

ubuntu@ubuntu:~$ sudo cryptsetup -c aes-xts-plain64 -s 512 -h sha512 luksFormat /dev/sda3

WARNING!
========
This will overwrite data on /dev/sda3 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter passphrase for /dev/sda3: 
Verify passphrase: 
ubuntu@ubuntu:~$ sudo cryptsetup luksOpen /dev/sda3 sda3_crypt
Enter passphrase for /dev/sda3: 
ubuntu@ubuntu:~$ ll /dev/mapper
total 0
drwxr-xr-x  2 root root      80 Apr 26 18:02 ./
drwxr-xr-x 21 root root    4520 Apr 26 18:02 ../
crw-------  1 root root 10, 236 Apr 26 17:44 control
lrwxrwxrwx  1 root root       7 Apr 26 18:02 sda3_crypt -> ../dm-0
ubuntu@ubuntu:~$ sudo pvcreate /dev/mapper/sda3_crypt 
  Physical volume "/dev/mapper/sda3_crypt" successfully created.
ubuntu@ubuntu:~$ sudo vg
vgcfgbackup    vgconvert      vgextend       vgmknodes      vgs
vgcfgrestore   vgcreate       vgimport       vgreduce       vgscan
vgchange       vgdisplay      vgimportclone  vgremove       vgsplit
vgck           vgexport       vgmerge        vgrename       
ubuntu@ubuntu:~$ sudo vgcreate system /dev/mapper/sda3_crypt
  Volume group "system" successfully created
ubuntu@ubuntu:~$ sudo lvcreate -n root -L 32G system
  Logical volume "root" created.
ubuntu@ubuntu:~$ ll /dev/mapper
total 0
drwxr-xr-x  2 root root     100 Apr 26 18:16 ./
drwxr-xr-x 22 root root    4560 Apr 26 18:16 ../
crw-------  1 root root 10, 236 Apr 26 17:44 control
lrwxrwxrwx  1 root root       7 Apr 26 18:16 sda3_crypt -> ../dm-0
lrwxrwxrwx  1 root root       7 Apr 26 18:16 system-root -> ../dm-1
ubuntu@ubuntu:~$ sudo lvcreate -n home -l 100%FREE system
  Logical volume "home" created.
ubuntu@ubuntu:~$ ll /dev/mapper
total 0
drwxr-xr-x  2 root root     120 Apr 26 18:16 ./
drwxr-xr-x 22 root root    4580 Apr 26 18:16 ../
crw-------  1 root root 10, 236 Apr 26 17:44 control
lrwxrwxrwx  1 root root       7 Apr 26 18:16 sda3_crypt -> ../dm-0
lrwxrwxrwx  1 root root       7 Apr 26 18:16 system-home -> ../dm-2
lrwxrwxrwx  1 root root       7 Apr 26 18:16 system-root -> ../dm-1

Running the installer

Once with the steps above, run the installer, and select partitioning option "something else". Just map code>BOOT, code>ROOT and code>HOME to the appropriate partition and logical volumes. Once the installer has run to completion, do not reboot, but return to the terminal.

Post installation steps

Unfortunately, installing Ubuntu this way would leave you with an unbootable system. The kernel itself and the initial ramdisk are located in the unencrypted BOOT partition. The initial ramdisk needs to be rebuilt in a way that it will be able to open the encryted LUKS volume and mount the logical volumes within it, the installer currently does not do that for us. The missing steps are editing one line in two files each, then running one command. The list of commands below puts these steps into context, starting with mounting the required file systems and entering a chroot from within which the few steps have to be carried out.


ubuntu@ubuntu:~$ sudo mount /dev/mapper/system-root /mnt
ubuntu@ubuntu:~$ sudo mount /dev/sda2 /mnt/boot
ubuntu@ubuntu:~$ sudo mount --bind /dev /mnt/dev
ubuntu@ubuntu:~$ sudo chroot /mnt
root@ubuntu:/# mount -t proc proc /proc
root@ubuntu:/# mount -t sysfs sys /sys
root@ubuntu:/# mount -t devpts devtps /dev/pts            
root@ubuntu:/# blkid /dev/sda3
/dev/sda3: UUID="01c3aa4e-5ed1-4ed7-9394-9c2ef3d85922" TYPE="crypto_LUKS" PARTLABEL="SYSTEM" PARTUUID="8b5f0ba1-c0e3-4072-91c3-b9c79ec4d8a8"
root@ubuntu:/# nano /etc/crypttab
root@ubuntu:/# cat /etc/crypttab
sda3_crypt UUID=01c3aa4e-5ed1-4ed7-9394-9c2ef3d85922 none luks,discard
             
root@ubuntu:/# nano /etc/cryptsetup-initramfs/conf-hook # set CRYPTSETUP=Y
root@ubuntu:/# grep ^CRYPTSETUP=  /etc/cryptsetup-initramfs/conf-hook
CRYPTSETUP=y
root@ubuntu:/# update-initramfs -c -k all
update-initramfs: Generating /boot/initrd.img-4.18.0-18-generic
W: Possible missing firmware /lib/firmware/nvidia/gv100/sec2/sig.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/sec2/image.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/sec2/desc.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/nvdec/scrubber.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_method_init.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_bundle_init.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_nonctx.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_ctx.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_sig.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_data.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_inst.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_bl.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_sig.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_data.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_inst.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_bl.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/ucode_unload.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/ucode_load.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/unload_bl.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/bl.bin for module nouveau
update-initramfs: Generating /boot/initrd.img-4.18.0-15-generic
W: Possible missing firmware /lib/firmware/nvidia/gv100/sec2/sig.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/sec2/image.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/sec2/desc.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/nvdec/scrubber.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_method_init.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_bundle_init.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_nonctx.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/sw_ctx.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_sig.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_data.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_inst.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/gpccs_bl.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_sig.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_data.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_inst.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/gr/fecs_bl.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/ucode_unload.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/ucode_load.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/unload_bl.bin for module nouveau
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/bl.bin for module nouveau
root@ubuntu:/# # ignore the warnings above
root@ubuntu:/# exit
exit
ubuntu@ubuntu:~$ # done

Once done, the system can be rebooted and the first steps of configuring the freshly installed Ubuntu 18.04 can be processed. These will be covered in the upcoming posts.