Clojure Datomic Exercise in IntelliJ IDEA / Cursive IDE.

Introduction

In my previous Integrant exercise, I had converted my earlier SimpleServer exercise to use the Integrant state management library. In that Integrant exercise, there were three datastores: in-memory datastore that read the initial data from CSV files, AWS DynamoDB datastore, and PostgreSQL datastore. I implemented the domain layer using Clojure Protocols so that in the application it was easy to switch the datastore by changing the value in one Integrant configuration (:backend/active-db ) - and reset the application state by using Integrant reset.

What is Datomic?

Datomic is an immutable transactional hosted database that uses Datalog language for queries. Being hosted means that Datomic uses another data store service, e.g. DynamoDB, for persistence.

Choosing Datomic Version

Choosing the Datomic version for development was a bit confusing: should I choose Datomic dev-local, Datomic Free, or Datomic Starter? Finally, I decided to use Datomic Starter.

Datomic Tooling

I created some Just recipes for Datomic development:

# Start local Datomic database.
@datomic:
cd datomic && ./start-datomic.sh
# Reset Datomic SimpleServer databases.
@datomic-ss-reset:
cd datomic && ./reset-simpleserver-db.sh
# Start Datomic Peer Server. NOTE: NOT NEEDED IN THIS EXERCISE!
@datomic-peer-server:
cd datomic && ./start-peer-server.sh
# Start Datomic Console.
@datomic-console:
cd datomic && ./start-console.sh
  • datomic-ss-reset: Deletes and creates simpleserver and simpleserver_test databases (for clean development start).
  • datomic-peer-server: Starts the Datomic Peer Server. Not needed in this exercise, I just tested that it works.
  • datomic-console: Starts the Datomic Console server, open Chrome: http://localhost:8080/browse
#!/bin/bash
THIS_DIR=$(pwd)
TRANSACTOR_DIR=/mnt/ssd2/local/datomic-pro-1.0.6202
pushd $THIS_DIR
cd $TRANSACTOR_DIR
pwd
sleep 2
bin/repl < $THIS_DIR/reset-simpleserver-db.clj
popd
(require '[datomic.api :as d])
(println "**********************************")
(println "Starting to reset Simpleserver databases...")
(def db-uri "datomic:dev://localhost:4334/simpleserver")
(def test-db-uri "datomic:dev://localhost:4334/simpleserver_test")
(println "Listing databases before reset...")
(d/get-database-names "datomic:dev://localhost:4334/*")
(println "Deleting databases...")
(d/delete-database db-uri)
(d/delete-database test-db-uri)
(println "Listing databases after deletes...")
(d/get-database-names "datomic:dev://localhost:4334/*")
(println "Creating databases...")
(d/create-database db-uri)
(d/create-database test-db-uri)
(println "Listing databases after creations...")
(d/get-database-names "datomic:dev://localhost:4334/*")
(def exercise-dir "/a/prs/github/clojure/webstore-demo/re-frame-demo")
(def ss-schema (read-string (slurp (str exercise-dir "/datomic/simpleserver-schema.edn"))))
(def ss-conn (d/connect db-uri))
(def test-ss-conn (d/connect test-db-uri))
@(d/transact ss-conn ss-schema)
@(d/transact test-ss-conn ss-schema)
...
(def product-groups (load-product-groups))
(def product-group-datoms (get-product-group-datoms product-groups))
@(d/transact ss-conn product-group-datoms)
...
Datomic Console.
(d/q '[:find ?title ?a_or_d ?year
:in $ ?pg-id ?y
:where
[?e :domain.product/title ?title]
[?e :domain.product/year ?year]
[?e :domain.product/a_or_d ?a_or_d]
[?e :domain.product/pg-id ?pg-id]
[(>= ?year ?y)]]
(d/db ss-conn) 2 2000)
#{["A History of Violence" "Cronenberg, David" 2005]
["Mulholland Dr." "Lynch, David" 2001]
["Paha maa" "Aku Louhimies" 2005]
["Avatar" "Cameron, James" 2009]
["Hyvä poika" "Zaida Bergroth" 2011]
...

Experimenting with the Datomic Sample

There are quite a lot of samples with the Datomic Starter installation package. Unfortunately, there was no standard deps.edn file with the samples. Therefore I added nrepl to the run script to be able to start a nrepl server and then connect to that server using IntelliJ IDEA / Cursive to make it easier to experiment e.g. with the Seattle sample that was provided with the installation package.

Setting Up the Connection to the Datomic Database

The connection to the Datomic database is created in the Integrant state management of the application:

:backend/datomic {:active-db #ig/ref :backend/active-db
:uri "datomic:dev://localhost:4334/simpleserver"}
(ns simpleserver.core
(:require
...
[datomic.api :as d]
...
(defmethod ig/init-key :backend/datomic [_ {:keys [active-db uri]}]
(log/debug "ENTER ig/init-key :backend/datomic")
(if (= active-db :datomic)
{:conn (d/connect uri)}))

Using the Datomic Api

Using the Datomic api was really easy and experimenting with the api using the Clojure REPL was a real joy. Some examples.

(def db-uri "datomic:dev://localhost:4334/simpleserver")
(d/create-database db-uri)
(def exercise-dir "/a/prs/github/clojure/webstore-demo/re-frame-demo")
(def ss-schema (read-string (slurp (str exercise-dir "/datomic/simpleserver-schema.edn"))))
(def ss-conn (d/connect db-uri))
@(d/transact ss-conn ss-schema)
[
; DOMAIN
; Product group
{:db/ident :domain.product-group/id
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one
:db/unique :db.unique/identity
:db/doc "The id of the product group"}
{:db/ident :domain.product-group/name
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one
:db/doc "The name of the product group"}
; Product
{:db/ident :domain.product/id
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one
:db/unique :db.unique/identity
:db/doc "The id of the product"}
...
...
(defn get-product-group-datoms [product-groups]
(mapv (fn [product-group]
(let [id (:id product-group)
name (:name product-group)]
{:domain.product-group/id id
:domain.product-group/name name}))
product-groups))
(def product-groups (load-product-groups))
(def product-group-datoms (get-product-group-datoms product-groups))
@(d/transact ss-conn product-group-datoms)
...
(credentials-ok?
[_ _ email password]
(log/debug (str "ENTER credentials-ok?"))
(let [found (d/q '[:find ?email ?hashed-password
:in $ ?email ?hashed-password
:where
[?e :user.user/email ?email]
[?e :user.user/hashed-password ?hashed-password]]
(d/db conn) email (str (hash password)))]
(if found
(= found #{[email (str (hash password))]})
false)))

Development Flow

The development flow using the Clojure REPL and Integrant was really nice. Usually, I experimented the queries in my scratch file (see more about this technique in my blog post Clojure Power Tools Part 1). Once I was confident the query worked I moved the code snippet from the scratch file to the production source file and reset Integrant state (with IntelliJ IDEA hotkey, of course), and ran the unit tests to see that the code worked also as part of the application.

Conclusions

It was really fun to do this exercise. The Day of Datomic video presentations were really good, also other material regarding Datomic and the Datalog query language. The exercise itself was quite effortless to implement using Datomic.

I’m a Software architect and developer. Currently implementing systems on AWS / GCP / Azure / Docker / Kubernetes using Java, Python, Go and Clojure.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store