Wednesday 3 July 2019

Ubuntu 18.04 on ThinkPad P50 (Part 2)

Corporate open client repository

One of the first steps was to add Open Client Debian Community (OCDC) package repository. This is a package repository which includes mandatory and optional packages provided by the corporation (well, maintained by our internal Linux community), including endpoint management tools that enable registration and compliance checks. Expect more on this in future posts.


tb@castor:~$ sudo apt-get install libfrontier-rpc-perl libjson-perl libgtk2-perl
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libcommon-sense-perl libjson-xs-perl libpango-perl libtypes-serialiser-perl
Suggested packages:
  libgtk2-perl-doc
The following NEW packages will be installed:
  libcommon-sense-perl libfrontier-rpc-perl libgtk2-perl libjson-perl
  libjson-xs-perl libpango-perl libtypes-serialiser-perl
0 upgraded, 7 newly installed, 0 to remove and 26 not upgraded.
Need to get 923 kB of archives.
After this operation, 4.237 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libcommon-sense-perl amd64 3.74-2build2 [20,3 kB]
Get:2 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libfrontier-rpc-perl all 0.07b4-6 [35,5 kB]
Get:3 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libpango-perl amd64 1.227-2build1 [157 kB]
Get:4 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libgtk2-perl amd64 2:1.24992-1build1 [544 kB]
Get:5 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libjson-perl all 2.97001-1 [73,3 kB]
Get:6 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libtypes-serialiser-perl all 1.0-1 [12,1 kB]
Get:7 http://hu.archive.ubuntu.com/ubuntu bionic/universe amd64 libjson-xs-perl amd64 3.040-1 [81,2 kB]
Fetched 923 kB in 1s (632 kB/s)           
Selecting previously unselected package libcommon-sense-perl.
(Reading database ... 161358 files and directories currently installed.)
Preparing to unpack .../0-libcommon-sense-perl_3.74-2build2_amd64.deb ...
Unpacking libcommon-sense-perl (3.74-2build2) ...
Selecting previously unselected package libfrontier-rpc-perl.
Preparing to unpack .../1-libfrontier-rpc-perl_0.07b4-6_all.deb ...
Unpacking libfrontier-rpc-perl (0.07b4-6) ...
Selecting previously unselected package libpango-perl.
Preparing to unpack .../2-libpango-perl_1.227-2build1_amd64.deb ...
Unpacking libpango-perl (1.227-2build1) ...
Selecting previously unselected package libgtk2-perl.
Preparing to unpack .../3-libgtk2-perl_2%3a1.24992-1build1_amd64.deb ...
Unpacking libgtk2-perl (2:1.24992-1build1) ...
Selecting previously unselected package libjson-perl.
Preparing to unpack .../4-libjson-perl_2.97001-1_all.deb ...
Unpacking libjson-perl (2.97001-1) ...
Selecting previously unselected package libtypes-serialiser-perl.
Preparing to unpack .../5-libtypes-serialiser-perl_1.0-1_all.deb ...
Unpacking libtypes-serialiser-perl (1.0-1) ...
Selecting previously unselected package libjson-xs-perl.
Preparing to unpack .../6-libjson-xs-perl_3.040-1_amd64.deb ...
Unpacking libjson-xs-perl (3.040-1) ...
Setting up libcommon-sense-perl (3.74-2build2) ...
Setting up libtypes-serialiser-perl (1.0-1) ...
Setting up libfrontier-rpc-perl (0.07b4-6) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up libjson-perl (2.97001-1) ...
Setting up libpango-perl (1.227-2build1) ...
Setting up libgtk2-perl (2:1.24992-1build1) ...
Setting up libjson-xs-perl (3.040-1) ...
tb@castor:~$ cd /tmp
tb@castor:/tmp$ mkdir ocdc
tb@castor:/tmp$ cd ocdc/
tb@castor:/tmp/ocdc$ wget http://ocdc.xxxxxxx.yyy.com/ocdc/{yyy-ca-certificates,ocdc-archive-keyring,ocdc-repository}.deb
tb@castor:/tmp/ocdc$ sudo dpkg -i *.deb

Minor adjustments to outdated package dependencies

Some of the community maintained corporate packages include dangling references to packages which are not available any more on Ubuntu 18.04. Fixing the package itself is the first step. The command listing below demonstrated the process on a prominent example: I lifted dependency on consolekit and libpam-ck-connector, which are not available any more, then rebuilt the package.


tb@castor:~$ cd /tmp/ocdc/
tb@castor:/tmp/ocdc$ apt-get download acme-security-compliance 
tb@castor:/tmp/ocdc$ dpkg-deb -x acme-security-compliance_5.0.4.1-12_all.deb acme-security-compliance
tb@castor:/tmp/ocdc$ dpkg-deb --control acme-security-compliance_5.0.4.1-12_all.deb acme-security-compliance/DEBIAN 
tb@castor:/tmp/ocdc$ cat acme-security-compliance/DEBIAN/control 
Package: acme-security-compliance
Version: 5.0.4.1-12
Architecture: all
Maintainer: OCDC Core Team <ocdc-approvers@ocdc.hursley.acme.com>
Installed-Size: 356
Depends: libpam-runtime, python, perl, libpam-passwdqc, ecryptfs-utils, acme-security-rules, gconf2, procps, sysv-rc, gnome-keyring, libpam-gnome-keyring, consolekit, libpam-ck-connector
Section: ACME/utils
Priority: extra
Description: Implementation of ACME security guidelines
 The ACME Network and Computing Security Services (NCSS) group has defined
 a set of mandatory security guidelines to protect ACME's information assets
 on network and computing environments within ACME.  This is referred to as the
 ITCS104 Security Standards for Providers of Network and Computing Services.
 In addition to ITCS104, there is a commercial Information Security
 Controls Document referred to as GSD331 which recommends
 additional guidelines for Strategic Outsourcing commercial customers.
 For additional information on these guidelines, visit the Network and
 Computing Security Services (NCSS) webpage at http://w3.security.acme.com
 The Client for e-business implements all of the guidelines from ITCS104 and
 the most commonly used elements of GSD331.  These settings are applied to
 the requisite files by installing patched versions.
tb@castor:/tmp/ocdc$ nano acme-security-compliance/DEBIAN/control 
tb@castor:/tmp/ocdc$ cat acme-security-compliance/DEBIAN/control 
Package: acme-security-compliance
Version: 5.0.4.1-12+dep1
Architecture: all
Maintainer: OCDC Core Team <ocdc-approvers@ocdc.hursley.acme.com>
Installed-Size: 356
Depends: libpam-runtime, python, perl, libpam-passwdqc, ecryptfs-utils, acme-security-rules, gconf2, procps, gnome-keyring, libpam-gnome-keyring
Section: ACME/utils
Priority: extra
Description: Implementation of ACME security guidelines
 The ACME Network and Computing Security Services (NCSS) group has defined
 a set of mandatory security guidelines to protect ACME's information assets
 on network and computing environments within ACME.  This is referred to as the
 ITCS104 Security Standards for Providers of Network and Computing Services.
 In addition to ITCS104, there is a commercial Information Security
 Controls Document referred to as GSD331 which recommends
 additional guidelines for Strategic Outsourcing commercial customers.
 For additional information on these guidelines, visit the Network and
 Computing Security Services (NCSS) webpage at http://w3.security.acme.com
 The Client for e-business implements all of the guidelines from ITCS104 and
 the most commonly used elements of GSD331.  These settings are applied to
 the requisite files by installing patched versions.
tb@castor:/tmp/ocdc$ gunzip acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian.gz
tb@castor:/tmp/ocdc$ nano acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian
tb@castor:/tmp/ocdc$ head acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian
acme-security-compliance (5.0.4.1-12+dep1) experimental; urgency=low

  * Don't replace the gdm-autologin file for pam. Autologin settings will
    be enforced by WST/TEM, and replacing it can break gdm.

 -- Tibor Bősze <tibor.boesze@hu.acme.com>  Fri, 03 May 2019 14:23:00 +0200

acme-security-compliance (5.0.4.1-12) experimental; urgency=low

  * Don't replace the gdm-autologin file for pam. Autologin settings will
    be enforced by WST/TEM, and replacing it can break gdm.
tb@castor:/tmp/ocdc$ gzip acme-security-compliance/usr/share/doc/acme-security-compliance/changelog.Debian
tb@castor:/tmp/ocdc$
tb@castor:/tmp/ocdc$ dpkg -b acme-security-compliance acme-security-compliance_5.0.4.1-12+dep1_all.deb
dpkg-deb: building package 'acme-security-compliance' in 'acme-security-compliance_5.0.4.1-12+dep1_all.deb'.

I added the +dep1 suffix to the package rather than increasing the package version. This is meant to indicate a local diversion and does not interfere with updates. Once can pin/hold a given version via apt.

Setting up a local apt repository

The next step was to create a local package repository which would contain all the locally brewed packages in the future. The main advantage is that I can still benefit from all the advantages apt offers compared to bare-foot dpkg. At some point, these can be shared or pushed to upstream after some review and approval process.


tb@castor:/tmp/ocdc$ cd /usr/local/lib
tb@castor:/usr/local/lib$ sudo mkdir apt
tb@castor:/usr/local/lib/apt$ sudo mkdir amd64
tb@castor:/usr/local/lib/apt$ sudo cp /tmp/ocdc/*dep1*.deb amd64/
tb@castor:/usr/local/lib/apt$ cd amd64
tb@castor:/usr/local/lib/apt/amd64$ apt-ftparchive packages . | sudo tee Packages
Package: acme-security-compliance
Architecture: all
Version: 5.0.4.1-12+dep1
Priority: extra
Section: ACME/utils
Maintainer: OCDC Core Team <ocdc-approvers@ocdc.hursley.acme.com>
Installed-Size: 356
Depends: libpam-runtime, python, perl, libpam-passwdqc, ecryptfs-utils, acme-security-rules, gconf2, procps, gnome-keyring, libpam-gnome-keyring
Filename: ./acme-security-compliance_5.0.4.1-12+dep1_all.deb
Size: 38272
MD5sum: c7103b33f23a302de24fa270f50f8532
SHA1: d41315e1faad5e39d33bfd8a86cd3ae54beef3be
SHA256: 643b621bf832e0438a4e18b3725c60a33c9164c1f15d7f29e7c21652944c312e
SHA512: 920cb177fde140a4f08262039015f0f67aaf785e177c2a31d3c22a5d6dd5e3b3636653330beac7ef1c7b17675b1d2a78226ff07a3d6eea1e4922f2430e3ce7e9
Description: Implementation of ACME security guidelines
 The ACME Network and Computing Security Services (NCSS) group has defined
 a set of mandatory security guidelines to protect ACME's information assets
 on network and computing environments within ACME.  This is referred to as the
 ITCS104 Security Standards for Providers of Network and Computing Services.
 In addition to ITCS104, there is a commercial Information Security
 Controls Document referred to as GSD331 which recommends
 additional guidelines for Strategic Outsourcing commercial customers.
 For additional information on these guidelines, visit the Network and
 Computing Security Services (NCSS) webpage at http://w3.security.acme.com
 The Client for e-business implements all of the guidelines from ITCS104 and
 the most commonly used elements of GSD331.  These settings are applied to
 the requisite files by installing patched versions.

tb@castor:/usr/local/lib/apt/amd64$ cd ..
tb@castor:/usr/local/lib/apt$ apt-ftparchive release . | sudo tee Release
Date: Fri, 03 May 2019 13:32:52 +0000
MD5Sum:
 46a0d423952446e8b31e4c8b2f6db4fa             1252 Release
 f1001d18fbe7ab32cab6178efff454a8             1619 amd64/Packages
SHA1:
 4faf1d629ba57672840f742c30d9f3520c94ae54             1252 Release
 cb820ae5264f24af1d3e1b1c1c0ac8d20e134224             1619 amd64/Packages
SHA256:
 5ef09f42467d1c89209f83fbeae48a722126e14519df72eca6d0b1c899493221             1252 Release
 612cb061581b2e1edfa480dd3569e6eddcbb85d30bb49170394d21d21c789fef             1619 amd64/Packages
SHA512:
 eac63ffee19c0a2e34556b209c1907e86b776515571291eb9b8e7fe435627815e6324e53fa8c1a2f044f0815546e5d3ae2e12ac8900ca89534c7bef1c006bfcd             1252 Release
 302fabb5e99825dc0951a9a3bbeb22be711058fe5a63a19e50b624e466608152bdff2694a398280fbc5101fc75a2d64fdd8eef712f9dc6e7d1a807aa17809d75             1619 amd64/Packages
tb@castor:/usr/local/lib/apt$ find .
.
./Release
./amd64
./amd64/acme-security-compliance_5.0.4.1-12+dep1_all.deb
./amd64/Packages
tb@castor:/usr/local/lib/apt$ sudo nano /etc/apt/sources.list
tb@castor:/usr/local/lib/apt$ tail /etc/apt/sources.list
deb http://security.ubuntu.com/ubuntu bionic-security main restricted
# deb-src http://security.ubuntu.com/ubuntu bionic-security main restricted
deb http://security.ubuntu.com/ubuntu bionic-security universe
# deb-src http://security.ubuntu.com/ubuntu bionic-security universe
deb http://security.ubuntu.com/ubuntu bionic-security multiverse
# deb-src http://security.ubuntu.com/ubuntu bionic-security multiverse

## local repo
deb [trusted=yes] file:/usr/local/lib/apt/amd64 ./

tb@castor:/usr/local/lib/apt/amd64$ sudo aptitude
Performing actions...

...
 
Press Return to continue, 'q' followed by Return to quit.

tb@castor:/usr/local/lib/apt/amd64$

Removing some snaps

Snappy is a software deployment platform developed by Canonical, introducing the snap package format, which is intended to be less dependent on the version of operating system and libraries. Unlike rpm or deb but very much like FlatPack and AppImage, Snappy intents to offer a modern application delivery model shipping self contained software packages which enable developers to publish and update their applications with less distribution specific overhead, and allow end users to receive the most up-to-date version of their application in a smooth manner.

Snappy has received much criticism, and while I believe there are valid use cases for deploying applications via snaps, I dislike the idea that Ubuntu 18.04 by default ships some core application - such as system-monitor - as snap. I prefer to keep the core of my environment using good-old debs. I decided to only deploy snaps as exception, if there is a strong compelling reason to do so. Therefore, I removed some pre-installed snaps and replaced them with their deb counterparts.


tb@castor:/tmp/ocdc$ snap list
Name                  Version         Rev   Tracking  Publisher   Notes
core                  16-2.37.1       6350  stable    canonical✓  core
gnome-3-26-1604       3.26.0          74    stable/…  canonical✓  -
gnome-calculator      3.30.1          260   stable/…  canonical✓  -
gnome-characters      3.30.0          139   stable/…  canonical✓  -
gnome-logs            3.30.0          45    stable/…  canonical✓  -
gnome-system-monitor  3.30.0          57    stable/…  canonical✓  -
gtk-common-themes     0.1-4-g88bc1b2  818   stable/…  canonical✓  - 
tb@castor:/tmp/ocdc$ sudo snap remove gnome-calculator gnome-characters gnome-logs gnome-system-monitor 
2019-05-03T13:31:49+02:00 INFO Waiting for conflicting change in progress...
2019-05-03T13:31:52+02:00 INFO Waiting for conflicting change in progress...
2019-05-03T13:31:58+02:00 INFO Waiting for conflicting change in progress...
2019-05-03T13:32:09+02:00 INFO Waiting for conflicting change in progress...
gnome-calculator removed
gnome-characters removed
gnome-logs removed
gnome-system-monitor removed
tb@castor:/tmp/ocdc$ sudo apt-get install gnome-calculator gnome-characters gnome-logs gnome-system-monitor 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libatkmm-1.6-1v5 libcairomm-1.0-1v5 libglibmm-2.4-1v5 libgtkmm-3.0-1v5
  libpangomm-1.4-1v5
The following NEW packages will be installed:
  gnome-calculator gnome-characters gnome-logs gnome-system-monitor
  libatkmm-1.6-1v5 libcairomm-1.0-1v5 libglibmm-2.4-1v5 libgtkmm-3.0-1v5
  libpangomm-1.4-1v5
0 upgraded, 9 newly installed, 0 to remove and 26 not upgraded.
Need to get 2.585 kB of archives.
After this operation, 15,5 MB of additional disk space will be used.
Do you want to continue? [Y/n] 

To be fair, let me restate that I do see valid use cases for deploying snaps. I deployed slack via Snappy. It is closed source, third party app and I prefer to use a more recent version than what a 5-years LTS will realistically provide, and run it isolated in it's own sandbox.