Use Clojure with Docker and Kubernetes!

Image for post
Image for post
Cursive REPL output window. Just listing the buckets using my repl scratch file.

Introduction

At my new company Metosin I have been implementing AWS infrastructure using Pulumi and implementing applications using Clojure. We are deploying applications into a Kubernetes cluster and to verify that our applications run with the infrastructure in a dockerized environment as they should we are running tests locally using Drone pipeline (see my previous blog post about it: Using Drone CI ) which spins the required infrastructure using docker containers (Postgres and Minio) and the actual application as another docker container. This week I was wondering that one application behaved a bit differently running tests in my local workstation hitting containers using localhost and running tests inside a docker container. I was discussing this with one Metosin colleague and he told me how to add an nrepl server inside the application so that I can connect to the application running in a docker container and experiment with the running application there using my IntelliJ IDEA / Cursive REPL. Because this trick was pretty neat I thought that I write a short blog post about it. (By the way, I haven’t regretted a second joining Metosin — one of the best places to be if you want to learn Clojure with the best clojurians on the planet, see more: My First Weeks at Metosin!)

Setting Up Nrepl Server Inside a Clojure Application

Our Clojure applications use Integrant as a state management library and we have our configuration in an Aero configuration file. Setting up an nrepl server inside the application is pretty straightforward using these libraries. First the configuration file:


:backend/nrepl {:bind #profile {:prod nil
:local “localhost”
:docker “0.0.0.0”
}
:port 3177}

(defmethod ig/init-key :backend/nrepl [_ {:keys [bind port]}]
(if (and bind port)
(nrepl/start-server :bind bind :port port)
nil))
(defmethod ig/halt-key! :backend/nrepl [_ this]
(if this
(nrepl/stop-server this)))
(defmethod ig/suspend-key! :backend/nrepl [_ this]
this)
(defmethod ig/resume-key :backend/nrepl [_ _ _ old-impl]
old-impl)

Docker Configuration

You can use e.g. Docker Compose when providing the docker configuration for your application. Example follows:

version: ‘3.7’
services:
arkisto_image:
image: local-arkisto:latest
ports:
— “3077:3077”
— “3177:3177”
networks:
[clojure-network]
container_name: arkisto_image
environment:
PROFILE: docker
DB_USERNAME: arkisto
DB_PASSWORD: arkisto
DB_HOST: clojuredb
DB_PORT: 5432
MINIO_HOST: clojures3
AWS_ACCESS_KEY_ID: minioadmin
AWS_SECRET_ACCESS_KEY: minioadmin
networks:
clojure-network:
name: clojure-network
; Endpoint only used in dev — default value that of Minio.
:endpoint #profile {:prod nil
:dev {:protocol :http
:hostname #or [#env “MINIO_HOST” “localhost”]
:port 9000}}

Connecting to the Nrepl Server From Your IDE

I use excellent Cursive REPL Clojure IDE as an example, but you can connect to the nrepl server pretty much same way using any Clojure IDE.

Image for post
Image for post
Cursive REPL configuration.
:aliases {
:kari {:extra-paths ["scratch"]
:extra-deps {hashp {:mvn/version "0.1.1"}
...
(require '[hashp.core])
(defn get-aws-credentials
"Get aws credentials. If AWS_PROFILE provided, use it. If AWS basic credentials, use them.
These are only for testing purposes. In real EKS environment do not provide any credentials
(returns nil) which means that pod uses role based authorization provided in the Pulumi
configuration."
[aws-profile aws-basic-credentials]
(let [{access-key-id :access-key-id
secret-access-key :secret-access-key} aws-basic-credentials]
(cond
aws-profile #p (credentials/profile-credentials-provider aws-profile)
(and access-key-id secret-access-key) #p (credentials/basic-credentials-provider aws-basic-credentials))))
Loading src/clj/arkisto/backend/main.clj... done
(integrant.repl/reset)
:reloading ()
#p[arkisto.backend.main/get-aws-credentials:39] (credentials/basic-credentials-provider aws-basic-credentials) => #<cognitect.aws.credentials$basic_credentials_provider$reify__23486@3c5cb0f5>
=> :resumed

How About If I Want to Connect to a Clojure Application inside Kubernetes Pod?

For connecting to a nrepl server running in an application running inside Kubernetes cluster I created a simple Justfile recipe so that my team members can easily connect to the applications:

# forward nrepl port from kube pod
@kube-nrepl +args='':
./infra/scripts/run-nrepl-port-forward.sh
...
MYNS="XXX-${MYAPP}-master" # provide namespace
MYPODNAME=$(kubectl get pods -n $MYNS -o=jsonpath='{.items[0].metadata.name}') # parse pod name
echo "Creating port forward from $MYPODNAME nrepl port $REMOTE_PORT to local port $MYPORT..."
kubectl -n $MYNS port-forward $MYPODNAME $MYPORT:$REMOTE_PORT # forward port
λ> just kube-nrepl arkisto 6677
Getting pod for arkisto...
Creating port forward from arkisto-master-897d9bd60c-6tm46 nrepl port 3177 to local port 6677...
Forwarding from 127.0.0.1:6677 -> 3177
Forwarding from [::1]:6677 -> 3177
; Sending form in repl connected to Docker...
(user/system)
=>
#:backend{:s3 #object[cognitect.aws.client.Client 0x432cd41f "cognitect.aws.client.Client@432cd41f"],
:bucket {:use-s3 nil,
:bucket "arkisto-mainsystem-bucket",
...
; Sending form in repl connected to Kubernetes...
(user/system)
=>
#:backend{:s3 #object[cognitect.aws.client.Client 0x37f83fa "cognitect.aws.client.Client@37f83fa"],
:bucket {:use-s3 true,
:bucket "my-very-secret-arkisto-bucket-xxxx", ; NOTE: bucket name changed in this blog!
...
...
; NOTE: Pulumi manages infra creation (S3 bucket).
(when-not use-s3
(aws/invoke s3-client {:op :CreateBucket :request {:Bucket bucket}}))
{:use-s3 use-s3 :bucket bucket :s3-client s3-client :clean clean})
...

Conclusions

If you want to learn an excellent functional language which is ready for the cloud era look no further: Clojure.

Written by

I’m a Software architect and developer. Currently implementing systems on AWS / GCP / Azure / Docker / Kubernetes using Java, Python, Go and Clojure.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store