Securing SSH with IDCS' OAuth service using the Device Code flow

May 9, 2019 | 11 minute read
Christopher Johnson
Director, Cloud Engineering
Text Size 100%:

In a blog post a couple of months ago I described how the OAuth Device flow works and gave some general and hypothetical examples of when you might use it. Just a couple of weeks ago I happened upon a real world use case for it and had a chance to put IDCS' Device Code support to actual use.

In most cases when you log into a Unix server over SSH you should be using a key pair - you upload your public key to the server and then the ssh client uses your private key to prove to the server that you are you. But there are times when you need to authenticate to a server that you haven't previously uploaded your public key to; for example if you provision Linux boxes with a bare OS you may not have user home directories. If those machines are facing the internet you definitely don't want your users authenticating via username + password, but you do need some way to safely authenticate them.

There are tons of different ways to accomplish that, but my customer specifically wanted users to authenticate with IDCS. And again, they didn't want to allow users to authenticate with just a username + password. As it turns out, the OAuth Device Code is perfectly suited for this use case for a whole bunch of reasons.

For example the Device flow can allow you to authenticate your Unix users with any mechanism supported by IDCS including:

  • Username + password
  • Multi factor authentication via a TOTP code, the Oracle Mobile Authenticator's Push feature, SMS, email or any other supported mechanism
  • Federation via SAML or Open ID Connect; which opens the door to biometric or any crazy authentication method you can come up with

Plus IDCS adds risk scoring to the login and SSO events, allowing you to deny access or force MFA in cases where the risk is higher than normal... for example someone who always logs in from Canada suddenly logging in from North Korea.

And it also "almost" supports SSO; meaning that if a user is already logged into IDCS when they SSH into a remote machine they need only click a link and then copy/paste a 6 character code into the browser window

The steps I took to get this working were:

  1. Create an Application in IDCS to represent the Unix boxes

  2. Install the Python PAM module

  3. Install a Python script that implements the Device flow in a PAM script

  4. Configure that script with IDCS' URL and the client ID from IDCS

  5. Test the PAM module and its config

  6. Configure OpenSSH to allow challenge / response authentication

  7. Configure SELinux to allow the PAM module to talk to IDCS

  8. Enable the PAM module for OpenSSH

For details on those steps check out the rest of the post below the fold!

 

Before I begin I want to thank Institute of Computer Science, Masaryk University and Ondrej Velisek upon whose work I based this post and code. It looks like Ondrej wrote the first implementation of these scripts as part of his PhD thesis. I initially forked his project but I wound up needing to rewrite it because I never got an answer from the Institute about the copyright on the code. So the only thing remaining from that code is the design, the config file format, and things like allowing for the use of a QR code.

Step 0: Update your IDCS user to include a Posix UID

Before you do anything else you need to have a user on your unix box with the same uid as a user in IDCS. In IDCS the POSIX User and Group IDs are stored in the urn:ietf:params:scim:schemas:oracle:idcs:extension:posix:User schema.

For example if your unix user looks like this in /etc/passwd:

cmj:x:1001:500::/home/cmj:/bin/bash

The matching user in IDCS should include this data:

"urn:ietf:params:scim:schemas:oracle:idcs:extension:posix:User": {
  "uidNumber": 1001
  "gidNumber": 500,
}
TIP: If you're creating a user just as a test remember to create them with a UID > 1000.

Step 1: Create an Application in IDCS

Every OAuth client needs to identify itself to IDCS. So you'll need to do the usual process:
  1. log into IDCS
  2. Click the + to create a new Application
  3. Choose Confidential as the application type - note that either Confidential or Mobile will work with the script I wrote, but there are security reasons to use Confidential.
  4. Give the app a name (I chose Unix Servers, but any name will do).
  5. Choose to Configure the app now.
  6. Check the box next to Device Code since that's the flow that the script implements.
  7. Under the Token Issuance Policy click the + under Grant the client access to Identity Cloud Service Admin APIs and select Me (I'll explain why in section 4)
  8. Then just next your way through and click Finish.
  9. Copy the Client ID and Secret into your notes - we'll need them in a bit.
  10. And finally don't forget to click Activate to actually activate the app!

Step 2: Install the Python PAM module

Rather than writing a PAM module in C or C++, I rely on a PAM module called pam-python that allows you to write Python scripts instead. pam-python provides a (mostly) direct Python binding to the PAM handle (including the environment) and saves you from dealing with C. Plus it shortens the whole code to test cycle by removing the compile + deploy step.

This step is different depending on which Linux distribution you're using.

Debian-derived distributions (including Ubuntu)

Debian makes this step easy. Python, the PAM module, and most of the dependencies we need for our script are available as packages and other can be installed via apt-get and pip. These two commands will get you where you need to be:
apt-get install -y python python-requests python-qrcode libpam-python python-pip pamtester
pip install pyyaml

RedHat-derived distributions (RedHat/CentOS/Oracle Linux/etc.)

RHEL derived distributions don't include the PAM module so you need to jump through an extra hoop or two.

First we need to make sure we have the Python interpreter and the Python libraries we need:

    # enable the EPEL package repo
    yum-config-manager --enable ol7_developer_EPEL
    
    # update anything that needs it
    yum -y update

    # then install the dependencies
    yum -y install python python-requests python-pip
    pip install qrcode pyyaml

The Python PAM module is not included in the public YUM repos but it is available pre-compiled in the usual places out on the internet. But since it's not distributed from the official OS sources it's probably not a good idea to just download it from some random site. Instead you should download the SRPM, review it carefully to make sure it looks trustworthy, and then build it yourself. I got my copy of the SRPM from the PKGS.org's Fedora repo.

3: Install a Python script that implements the Device flow in a PAM script

As I mentioned above, I began with a script written by someone else. You can begin with the same code or write your own.

I'm currently still working my way through the process of getting approval to release my code as is. I'll revise this post with a link to it once that's done but in the mean time you can easily recreate what I did.

The basic gist of the script is pretty straightforward - in the "pam_sm_authenticate():" function you do the normal OAuth Device call to acquire a user code. IDCS doesn't (as of this writing) support the verification_uri_complete feature so you'll need to present the URL (verification_url) and the user code (user_code) and instruct the user to copy/paste that code.

NOTE: During this work I also discovered a curious issue in the interaction between PAM and sshd. The PAM API allows a PAM module to send a message to the user by calling the pamh.conversation(pamh.Message(pamh.PAM_TEXT_INFO, "some text")). But sshd doesn't support that - if you try it will simply drop the TCP connection on you. So instead of PAM_TEXT_INFO you need to use PAM_PROMPT_ECHO_ON or PAM_PROMPT_ECHO_OFF and ask the user to press enter.

Once the user presses Enter you enter the normal OAuth Device code polling loop.

Finally, after the authentication has been completed there's a final check that you need to perform. The user code that the PAM module presented to the user isn't in any way related to the username of the user logging into the unix system. So we need to make sure that the Unix user and the IDCS user are the same by ourselves. Further complicating matters is the fact that a user's IDCS username is only rarely (meaning almost never) the same as the Unix username; this is because, by default, the IDCS username is an email address. IDCS anticipated this issue and includes a spot in the user schema for the Unix UID - in fact we set that up properly in Step 0 above.

The script needs to get that information from IDCS via the Me endpoint. Then it simply compares the value from IDCS to the value from pwd.getpwnam( pamh.user ).pw_uid which handles the complexity off whatever system you use for user identity - /etc/passwd, NIS, LDAP, or whatever system your have configured in nsswitch.conf.

And finally, don't forget to implement the other functions:

# Need to implement all methods to fulfill pam_python contract

def pam_sm_setcred(pamh, flags, argv):
    return pamh.PAM_SUCCESS


def pam_sm_acct_mgmt(pamh, flags, argv):
    return pamh.PAM_SUCCESS


def pam_sm_open_session(pamh, flags, argv):
    return pamh.PAM_SUCCESS


def pam_sm_close_session(pamh, flags, argv):
    return pamh.PAM_SUCCESS


def pam_sm_chauthtok(pamh, flags, argv):
    return pamh.PAM_SUCCESS

 

Step 4: Configure that script with IDCS' URL and the client ID from IDCS

The config.yml file has a field for the OAuth Client ID and Secret (along with IDCS' URL and a bunch of other stuff). You need to take the client ID and secret created above in step 1 and insert them there.
oauth:
    client:
        id: 123497812378468172348691346
        # client_secret required only when the client is configured as Confidential in IDCS
        secret: asdfkj-adfkj-ert9fdg
    scope: [openid approles groups urn:opc:idm:__myscopes__]
    device_endpoint: https://idcs-xxx.identity.oraclecloud.com/oauth2/v1/device
    token_endpoint: https://idcs-xxx.identity.oraclecloud.com/oauth2/v1/token
    userinfo_endpoint: https://idcs-xxx.identity.oraclecloud.com/admin/v1/Me?attributes=urn%3Aietf%3Aparams%3Ascim%3Aschemas%3Aoracle%3Aidcs%3Aextension%3Aposix%3AUser%3AuidNumber,urn%3Aietf%3Aparams%3Ascim%3Aschemas%3Aoracle%3Aidcs%3Aextension%3Aposix%3AUser%3AgidNumber
In this case I created the application as a Public Client and so the secret is left commented out. If you created the App as a Confidential Client you'd need to uncomment that and insert the secret after the colon. Also remember that the file is YAML, so you have to be careful about indentation.

The other fun fact you'll notice is that the userinfo_endpoint isn't pointing to the normal OIDC user info URL published in IDCS' OIDC metadata, and is instead pointing to the Me endpoint. As I mentioned above the PAM script needs to get the POSIX UID from IDCS to make sure that the user you are trying to log into the Unix box with is the person as you've logged into IDCS. Unfortunately IDCS' OpenID Connect User Info endpoint doesn't return that value (for a number of reasons). And so we are using the Me endpoint instead. I'm sure it seems strange that I would name the parameter "userinfo_endpoint" when I'm always going to point it to the Me endpoint, but that was the name in the original script and I was trying to preserve that file format as much as possible.

NOTE: I will likely change the script in the future to use the OIDC Discovery but for now this is how it works.

Step 5: Test the PAM module and its config!

Before you configure the system to use the module + script it's probably a good idea to test it.

Because if something's wrong you won't be able to log in again to fix it (at least not without jumping through some hoops)!

Enable the pamtester PAM config

Included in my source code is a file named "pamtester". That file contains the following:
auth required pam_python.so pam_oauth2_device.py
Copy that file to /etc/pam.d/pamtester or just create that file with the above content.

What this file does is tells the PAM subsystem that if an application asks to authenticate (auth) as the pamtester service that it should use the pam_python.so library and pass it the argument pam_oauth2_device.py which is the name of our script.

Run pamtester

Run the pamtester command with the following arguments:
  • -v
  • pamtester
  • the username
  • authenticate
So something like:
pamtester -v pamtester cmj authenticate
This will tell pamtester to be verbose, use the PAM service pamtester, the username you provide, and ask it to authenticate the user.

You should see something like this:

pamtester output

Now, go ahead and open that URL in your browser and then enter the code.

Once you've done that, come back to the terminal and hit enter.

You should see "pamtester: successfully authenticated".

If you do then you're ready to continue. If not you're going to need to troubleshoot the error you got back.

Step 6: Configure OpenSSH to allow challenge / response authentication

By default on most Linux systems the openssh daemon is not configured to allow challenge / response authentication. And so when the PAM module says "ok challenge the user with this text" the openssh daemon simply drops the connection; note that this is different from the issue I mentioned under step 3, if you leave OpenSSH at the default configuration it will drop the connection whether you try to prompt or just send text info to the user.

Edit your sshd config (almost always /etc/ssh/sshd_config) and find the ChallengeResponseAuthentication line and make sure it's set to yes.

It should look like this:

ChallengeResponseAuthentication yes

If you want your users to be able to log in with either their public key or using OAuth Device Code, and you haven't already enabled public key authentication now would probably be a good time to do that too. Find any existing line starting with "PubkeyAuthentication" and change the value there to "yes".

And Make sure that UsePAM is set to yes.

So your three values should be:

ChallengeResponseAuthentication yes
PubkeyAuthentication yes
UsePAM yes

If you want your users to only be able to log in with the OAuth Device code then you should set "PubkeyAuthentication no".

NOTE: As always if you aren't 100% sure what this all does please don't do this on a production server!

Step 7: Configure the SELinux to allow the PAM module to talk to IDCS

If you are using SELinux (i.e. if "SELINUX=enforcing" in /etc/selinux/config) then you will need to adjust your policy to allow the PAM modules to open TCP socket connections to IDCS.

There are a number of ways to do this, but my preferred is option 3 below.

Option 1: Edit /etc/selinux/config

In there change SELINUX=enforcing to SELINUX=permissive and then reboot. There are ways to change SELinux' config without rebooting but this is the best way to know you got it right and it won't break the next time you reboot.

Option 2: Create your own policy

For example you can use the audit2allow script that's part of the policycoreutils-python package on Oracle Linux / RedHat / CentOS

Option 3: be lazy and let someone else do the work

On RedHat and most other flavors of Linux there's an already existing SELinux policy intended for when NIS is enabled that allows login to make calls out to remote servers. Instead of disabling SELinux or figuring out how to write our own policies to allow the outbound TCP connections we can just use the existing policy that already does that for us.

To enable that policy permanently just run:

setsebool -P nis_enabled 1

If you ever need to turn it back off you need only re-run the command and change the 1 to a zero.

Step 8: Enable the PAM module for OpenSSH

Finally update your PAM config so that the sshd server uses the new module

Up above I provided a sample configuration for the pamtester service. Unfortunately I can't give you the same for the sshd service because its configuration is more complicated and differs depending on a bunch of stuff.

But most people will be fine just adding this:

auth sufficient pam_python.so pam_oauth2_device.py /etc/pam_oauth2_device/config.yml

If you're using a version of Linux like the ones I've tested on then that should go directly above auth substack password-auth. Check the PAM man pages for more info.

Remember, to make sure your config is correct you can always use "pamtester -v sshd USERNAME authenticate" to test.

What does it all look like?

If you've done everything above when you use ssh to connect you should see something like this:

Demonstration of OAuth Device flow to SSH

Or, in text form:

  1. You connect to the server over ssh - sending your unix username as part of the initial handshake
  2. The server displays some instructions, a URL to open, a code to enter, and possibly a QR code (rendered in ASCII)
  3. You open that URL in your browser and enter the code
    NOTE: In the image above I was already signed in. If you weren't yet signed in you would need to do that.
  4. After you enter the code and click Submit you switch back to the ssh window
  5. Finally hit enter to complete the login
  6. After that Bob's your uncle!

 

Christopher Johnson

Director, Cloud Engineering

Former child, Admiral of the bathtub navy, noted author and mixed medium artist (best book report, Ms Russel's 4th grade class, and macaroni & finger paint respectively), Time Person of the Year (2006), Olympic hopeful (and I keep hoping), Grammy Award winner (grandma always said I was the best), and dog owner.


Previous Post

Setting up an RDF Graph Database in Oracle Cloud Infrastructure (OCI)

Emma Thomas | 4 min read

Next Post


Oracle Cloud Infrastructure Compartments

Andre Correa Neto | 7 min read