Tuesday, 8 January 2013

Lotus Notes migration to Ubuntu 12.10 64bit

Having made the first steps in securing the W530 (see previous posts) and having set up further security & compliance related components as dictated by corporate policy, I was ready to start migrate my data from the old t400 thinkpad to the new box. The most critical part of the data to be migrated is Lotus Notes data. I am proficient with the operating system and all other software components I have on my old laptop but I am not a Lotus Notes experts at all, it is mostly a black box for me and a potential source of trouble challenges. The old laptop is running Ubuntu 10.04 32bit while I set up the new one with Ubuntu 12.10 64bit as mentioned in previous posts.

Backing up Lotus Noted Data

To clarify the scope, I was migrating from my data from a Lotus Notes 8.5.2 environment on Ubuntu 32bit to a 8.5.3 one on 64bit. Additionally, I also decided to clean the cruft accumulated over the last 7 years... I checked for defunct workspace icons and any obsolete local replicas and removed those. Then sorted all email in my inbox to their appropriate folders (usually, I do this while waiting for connecting flights at airports) and archived all emails from before 2013. I also changes the archive settings to save any new items to a new archive file. After cleaning up, I was ready to migrate my data.

First and foremost, do know what you have to migrate. I wanted to migrate the minimum required amount of settings but not less. Lotus Notes stores data in the proprietary Network Storage Facility format, these files have nsf os NSF extension. These files are self contained document-oriented databases storing semi structured data - the application logic and view definitions are also incorporated. There are some files like cache.nsf or log.nsf that do not carry valuable information, others will have to be backed up.

My mail file is just a normal nsf file, however it is encrypted with the private key stored in my ID file. It is a good practice to keep a backup of your ID file anyway, I have seen people believe that remembering their Lotus Notes password would be enough to recover a corrupt or deleted ID file...

Workspace definitions and icons are stored in the file desktop8.ndk, which also needs to be backed up. I also made sure to backup my custom signatures.

There are some settings stored in the eclipse workspace, but I did not migrate any of those. I was trying hard to start with an clean as possible environment and ended up with using the following command to create the backup on the old machine:


# backup all databases except log.nsf, the id file, desktop file and signatures
find ~/lotus/notes \( -iname '*.nsf' -o -iname '*.id' \
  -o -iname '*.htm' -o -iname 'desktop8.ndk' \) \
  -not -iname 'log.nsf' -print0 | tar cspzf \
/media/SAMSUNG/backup/t400-20130103/lotus_notes_data-sparse.tar.gz --null -T -

Installation and migration

On the W530, before restoring the files, I installed Lotus Notes and let it create the data directory, and initialize the ~/lotus/notes/data folder with default files. To do so, I started it, waited until the splash screen disappears and the first application window appears asking me inputs for initial setup. At this very first screen, I pressed cancel to abort and exit. I made sure all Lotus Notes processes terminated - one can use the "Lotus Notes Zap" utility for this.

Then I extracted the data and issues some more tweaks using the commands listed below. Other minor customization like setting the theme and default font was done from within the graphical interface, but nothing worth to document...


cd ~ # make sure we are in the right directory
tar xzf /media/tibi/SAMSUNG/backup/t400-20130103/lotus_notes_data-sparse.tar.gz
# start Lotus Notes... and choose "office network" as current location, look around, then exit.

chmod -x ~/lotus/notes/data/*.gif # just some cosmetics...
# I use English language, but more with my preferred date format
cat <<EOF >> ~/lotus/notes/data/notes.ini
DateOrder=YMD
ClockType=24_HOUR
DateSeparator=-
TimeSeparator=:
EOF
# restart notes and verify the date format used in your inbox.

The result is a fully functional install with all the data migrated, however it is lacking a few more tweaks to improve the visual appearance of native widgets. These tweaks are going to be covered in the next post.

5 comments:

  1. Just installed Notes 9 on Ubuntu 64 without problems.
    Maybe upgrading to this version will solve all problems that I have read about the 8.5.x client.

    http://wp.me/p1CuQM-hQ

    For reading on how to

    ReplyDelete
    Replies
    1. My post merely focuses on data migration - that is, importing stuff after installing from scratch, which I believe is no different for version 9.

      Having read your post the phrase 'without problems' seems a bit strange. Requiring manual changes to debian control files is not rocket science but still outside of what most would call an installation without problems. Anyway, your article is a nice step-by-step description install procedure - I have been helping a colleague of mine through the very same process a few months ago and your post would have saved me some time and head-scratching, but it did not exist at that time.

      One side note: I believe many would misunderstand your nomenclature on "recompilation of packages". In fact you were editing the control files and repacking/compressing the contents (deb packages are 'ar' archives - http://en.wikipedia.org/wiki/Deb_%28file_format%29) whereas recompiling a package usually refers to the process of building/compiling from source. And given that Lotus Notes is proprietary, recompilation will only be done by the Big Blue.

      Thanks for sharing the link!

      Cheers,
         T

      Delete
    2. This comment has been removed by the author.

      Delete
    3. I did change nomenclature.
      Thanx for input.
      Without problems is linked to my conclusion that installing anything in a corporate environment always needs to be customized anyway.
      I provided a sample of that.
      Please give me more input, because I think is bad for the platform not supporting Ubuntu 64 out of the box.

      Cheers
      Mats

      Delete
    4. Actually, I would very much expect a native 64bit Lotus Notes as opposed to the current 32bit version, but this is outside of what we can achieve.

      One more suggestion: whenever you customize a debian package, changing the version and indicating the changes you applied in the changelog is a good practice, one example is here: http://tuxicate.blogspot.hu/2013/01/optimus-and-ubuntu-1210-part-4.html

      Further, in a corporate environment, you should have a local apt-mirror anyway, and you can easily add your customized golden package to this local apt repository. This would allow for automated rollout to any number of clients and save a lot of bandwidth as other debian packages would also have to be downloaded only once to the mirror, and then distributed across the LAN locally. And this way, dependencies or you custom package could be installed automatically.

      If you are targeting a corporate environment with more than 20 clients, then I would recommend this approach.

      Cheers,
         T

      Delete