How to install packages on Fedora CoreOs (or Install the AWS SSM Agent)

TL;DR: As is traditional I question and complained about the chosen linux distribution only to discover a new linux skill to make my OS infrastructure immutable, be able to easily rollback or revert (deploy) to specific OS configurations states.

I want to use AWS Systems Manager Sessions Manager so I can remotely connect to my EC2 instance hosting my Forem server without having to use or manage an SSH key.

In order to do that I will need the SSM Agent install.

Most Cloud Service Providers (CSPs) have supported distributions of Linux which are security hardended by the CSPs and will come with some agents installed so you can start leveraging native cloud services like AWS Systems Manager.

Forem chose to use Fedora, and unfortunately no CSP supports Fedora, meaning you will have to do your own work to translate the official CSPs documentation when installing packages and agents.

If you are building a product around a VM to be cloud-agnonstic I would recommend using CentOS because for some reason all the CSPs have settled on this OS. CentOS is similar to Fedora in that they both derive from RedHat Linux.

Translating Linux versions

Fedora, CentOS and Amazon Linux 2 (AML2) all use the same package manager. The Redhat Package Manager, so if we can roughly translate our Fedora version to Redhat Version we can figure out what package to install.

Fedora versioning goes from 1 to 34.

Redhat versioning goes from 1 to 8

Fedora is using the upstream of Redhat meaning that its versions a faster rate with more upto date package. If you have to use an older version you will need to do some Google-fu to translate the version over.

If you are using nearly the latest version of Fedora then its going be the latest version of Redhat.

So when you go to install the SSM agent on AWS documentation you are going want to follow the RHEL 8 instructions.

So we want to install the Agent via this command:

sudo dnf install -y https://s3.us-east-1.amazonaws.com/us-east-1/latest/linux_amd64/amazon-ssm-agent.rpm

However.... I was presented with this error sudo: dnf: command not found

The package manager for RedHat has been traditionally been yum and dnf a newer version called Dandified YUM. DNF is suppose to be present on Fedora 28 and greater. So its odd its not running.

So we swap out dnf for yum as sanity check...

sudo yum install -y https://s3.us-east-1.amazonaws.com/us-east-1/latest/linux_amd64/amazon-ssm-agent.rpm

and its telling me yum cannot be found.... sudo: yum: command not found

Okay, so lets use which command to see if they exist. Lets do a sanity check for with python first:

[core@www (www.thedev.cloud) ~]$ which python
/usr/local/bin/python

Now yum and dnf

[core@www (www.thedev.cloud) ~]$ which yum
/usr/bin/which: no yum in (/var/home/core/.local/bin:/var/home/core/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)
[core@www (www.thedev.cloud) ~]$ which dnf
/usr/bin/which: no dnf in (/var/home/core/.local/bin:/var/home/core/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)

On Fedora 28 and greater, Yum is aliased to Dnf, so you can interchangeably use both commands.

So they aren't installed. hmm...

So after some Google soul searching I think Forem using a variant of Fedora called Fedora Atomic and can I see why they chose this variant of linux because the Atomic Project gives you:

• immutable infrastructure
• delivery of atomic or incremental updates and easy rollback (in case an upgrade goes wrong)

So essentially it is helping us enforce best practices by not letting us just meddle around on the machine.

RPM-OS Tree

So the way we install packages is not through yum or dnf but through rpm-ostree.

RPM-OStree is a hybrid image/package system and the easiest way to think of it is that it is a Git for your OS.

Running rpm-ostree status will shows us the installed layers:

● ostree://fedora:fedora/x86_64/coreos/stable
                   Version: 34.20210626.3.1 (2021-07-14T14:49:01Z)
                BaseCommit: 252fffde6f56d183a3c51c05a0c602b61011f6cb4de23a58313ba3b0023dc360
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
           LayeredPackages: systemd-oomd-defaults

  ostree://fedora:fedora/x86_64/coreos/stable
                   Version: 34.20210626.3.1 (2021-07-14T14:49:01Z)
                    Commit: 252fffde6f56d183a3c51c05a0c602b61011f6cb4de23a58313ba3b0023dc360
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39

So to install SSM Agent we use:

sudo rpm-ostree install https://s3.us-east-1.amazonaws.com/amazon-ssm-us-east-1/latest/linux_amd64/amazon-ssm-agent.rpm

If we were to run rm-ostree status again we can see our new layer

  ostree://fedora:fedora/x86_64/coreos/stable
                   Version: 34.20210626.3.1 (2021-07-14T14:49:01Z)
                BaseCommit: 252fffde6f56d183a3c51c05a0c602b61011f6cb4de23a58313ba3b0023dc360
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
                      Diff: 1 added
           LayeredPackages: systemd-oomd-defaults
             LocalPackages: amazon-ssm-agent-3.0.1390.0-1.x86_64

● ostree://fedora:fedora/x86_64/coreos/stable
                   Version: 34.20210626.3.1 (2021-07-14T14:49:01Z)
                BaseCommit: 252fffde6f56d183a3c51c05a0c602b61011f6cb4de23a58313ba3b0023dc360
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
           LayeredPackages: systemd-oomd-defaults

  ostree://fedora:fedora/x86_64/coreos/stable
                   Version: 34.20210626.3.1 (2021-07-14T14:49:01Z)
                    Commit: 252fffde6f56d183a3c51c05a0c602b61011f6cb4de23a58313ba3b0023dc360
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39

If we wanted to rollback on this installation we can do a rpm-ostree rollback

If wanted to deploy a specific version we can do rpm-otree deploy 34.20210626.3.1

If you want to learn more you give the RPM-OS-tree more of a read:

https://coreos.github.io/rpm-ostree/

Hot tip from Joe 🔥

Fedora Atomic was the old OS that got "merged" with CoreOS Container Linux when Red Hat bought CoreOS, Inc

So to clarify Fedora Atomic + CoreOS Container Linux = Fedora CoreOS

Late Observation

When you SSH in it may also tell you the linux version. I never noticed this before, but it clearly tell you its Fedora CoreOs and its version. 🤦 🤦🤦

~/Sites/selfhost[main]: ssh core@54.164.107.192
Fedora CoreOS 34.20210626.3.1

Conclusion

While we would get synergies using supported cloud linux distributions like CentOS or Amazon Linux 2, the trade off using Fedora Atomic means I never have to worry about what was installed, how to uninstall something, or how to get my OS into a specific state of configuration.

It makes me question why Cloud Service Providers (CSPs) don't use a flavour of linux that is atomic.