|
@@ -32,131 +32,6 @@ A baseline Clojure stack that runs on any and all platforms, that is nice to dev
|
|
|
Much influence has been drawn from Continuous Delivery, The Reactive Manifesto, and the Lean Startup. Talks such as
|
|
|
[Turning the Database Inside out]() and ___ have also had large influence.
|
|
|
|
|
|
-
|
|
|
-### Aggregate Objects
|
|
|
-
|
|
|
-After two years of clojure and data stream programming, one begins to think of all objects as aggregates of the
|
|
|
-events that it they have experienced. What is a key/value store? A sequence of keys and values. How can we build one?
|
|
|
-
|
|
|
-1. `(write {:key "harlan.name.first" :value "Harlan"})`, we'll use `[harlan.name.first Harlan]` for brevity.
|
|
|
-2. `[harlan.name.last Iverson]`
|
|
|
-3. `[harlan.job Hair Stylist]`
|
|
|
-4. `[alice.name.first Alice]`
|
|
|
-5. `[alice.name.last Simply]`
|
|
|
-6. `[alice.job Programmer]`
|
|
|
-
|
|
|
-```
|
|
|
-{"harlan.name.first" "Harlan"
|
|
|
- "harlan.name.last" "Iverson"
|
|
|
- "harlan.job" "Hair Stylist"
|
|
|
- "alice.name.first" "Alice"
|
|
|
- "alice.name.last" "Simply"
|
|
|
- "alice.job" "Programmer"}
|
|
|
-```
|
|
|
-
|
|
|
-7. `[harlan.job Programmer]`
|
|
|
-
|
|
|
-```
|
|
|
-{"harlan.name.first" "Harlan"
|
|
|
- "harlan.name.last" "Iverson"
|
|
|
- "alice.name.first" "Alice"
|
|
|
- "alice.name.last" "Simply"
|
|
|
- "alice.job" "Programmer"
|
|
|
- "harlan.job" "Programmer"}
|
|
|
-```
|
|
|
-
|
|
|
-The rule here is to have one slot per key that can hold a value, and keep the latest one we receive; remember,
|
|
|
-these are write events. Thankfully Computer Science gave us the Dictionary data structure that we can leverage:
|
|
|
-
|
|
|
-```clojure
|
|
|
-(def assoc clojure.core/assoc) ; not needed, just for example. works on dictionaries, ka maps.
|
|
|
-
|
|
|
-(defn kv-store []
|
|
|
- (let [store (atom {})]
|
|
|
- (fn [event] (swap! store assoc (:key event) (:value event))))
|
|
|
-```
|
|
|
-
|
|
|
-We can image more interesting objects, like summation or moving average. I am using objects more in the logic sense than
|
|
|
-the OOP sense.
|
|
|
-
|
|
|
-```clojure
|
|
|
-(defn- inc [value] (+ 1 (or value 0)))
|
|
|
-
|
|
|
-(defn sum []
|
|
|
- (let [value (atom 0)]
|
|
|
- (fn [event] (swap! value inc)))
|
|
|
-```
|
|
|
-
|
|
|
-```clojure
|
|
|
-(defn- cma-pair [[i avg] value]
|
|
|
- (let [new-i (+ 1 i)
|
|
|
- new-avg (/ (+ avg value) new-i)]
|
|
|
- [new-i new-avg]))
|
|
|
-
|
|
|
-(defn cma []
|
|
|
- (let [state (atom [0 0])]
|
|
|
- (fn [event] (second (swap! state update cma-pair (:value event))))))
|
|
|
-```
|
|
|
-
|
|
|
-
|
|
|
-These are foundational events. Let's try something more advanced:
|
|
|
-
|
|
|
-
|
|
|
-```clojure
|
|
|
-(defn cma []
|
|
|
- (let [state (atom [0 0])]
|
|
|
- (fn [event] (second (swap! state update cma-pair (:value event))))))
|
|
|
-```
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
-Stream Middleware.
|
|
|
-
|
|
|
-https://github.com/pyr/riemann-kafka
|
|
|
-
|
|
|
-Riemann + Kafka...
|
|
|
-
|
|
|
-
|
|
|
-http://allthingshadoop.com/2014/04/18/metrics-kafka/
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
-These aggregate objects can be composed using Clojure's inbuilt `(comp)` function.
|
|
|
-
|
|
|
-```
|
|
|
-;
|
|
|
-
|
|
|
-(defrecord EventsByCountry [users number-of-users]
|
|
|
- (init []
|
|
|
- {:users (atom (or users (kv-store)))
|
|
|
- :number-of-users (atom (or number-of-users (count)))})
|
|
|
- (process [forward events event]
|
|
|
- (swap! )))
|
|
|
-
|
|
|
-(let [users (kv-store)
|
|
|
- number-of-users (count)]
|
|
|
- (do (users event)
|
|
|
- (number-of-users event)))
|
|
|
-```
|
|
|
-
|
|
|
-
|
|
|
-They can then be placed in pipelines as xforms on core.async channels.
|
|
|
-
|
|
|
-```
|
|
|
-;
|
|
|
-
|
|
|
-(def input (chan))
|
|
|
-
|
|
|
-(def
|
|
|
-
|
|
|
-
|
|
|
-```
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
### Claims
|
|
|
|
|
|
|
|
@@ -174,18 +49,6 @@ They can then be placed in pipelines as xforms on core.async channels.
|
|
|
## Choices
|
|
|
|
|
|
|
|
|
-### Distributed Authorization
|
|
|
-
|
|
|
-- Each entity has an identity.
|
|
|
-- Unauthenticated entities have auto-generated identities until they authenticate.
|
|
|
-
|
|
|
-Mutator component
|
|
|
-
|
|
|
-- Drop into debug mode for live sessions.
|
|
|
-- Push app updates.
|
|
|
-- Access controlled... grant entities access.
|
|
|
--
|
|
|
-
|
|
|
### Clojure
|
|
|
|
|
|
The fundamental data structures representing code and data; nothing more, nothing less.
|
|
@@ -422,6 +285,12 @@ This means that we have an endpoint for iOS, Android, Web. Core logic is require
|
|
|
may vary slightly if only in deployment schedule.
|
|
|
|
|
|
|
|
|
+### Distributed Authorization
|
|
|
+
|
|
|
+- Each entity has an identity.
|
|
|
+- Unauthenticated entities have auto-generated identities until they authenticate.
|
|
|
+
|
|
|
+
|
|
|
|
|
|
### Ansible
|
|
|
|
|
@@ -502,14 +371,9 @@ https://github.com/liebke/avout
|
|
|
|
|
|
|
|
|
|
|
|
+## Design
|
|
|
|
|
|
-## Thanks
|
|
|
-
|
|
|
-Thanks to all the authors and contributors of the projects used and literature referenced.
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
-## Different types of compnents
|
|
|
+### Different types of compnents
|
|
|
|
|
|
Think of a component as a binary, and channels as standard in/out. Some binaries run on servers, others on phones, others
|
|
|
on desktop computers, etc. Each has different capabilities, a common one being POSIX. When you write a component in Clojre,
|
|
@@ -557,6 +421,16 @@ you think in terms of the capabilities of a system. Component helps us manage th
|
|
|
|
|
|
|
|
|
|
|
|
+Mutator component
|
|
|
+
|
|
|
+- Drop into debug mode for live sessions.
|
|
|
+- Push app updates.
|
|
|
+- Access controlled... grant entities access.
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Remote Atom
|
|
|
+
|
|
|
Idea:
|
|
|
|
|
|
- routing for subkey balancing -- atomic in ZK w/ zk-atom
|
|
@@ -566,6 +440,8 @@ Idea:
|
|
|
- subscribe with a predicate, channel is a special case. #(= (:channel %) "my-channel")
|
|
|
|
|
|
|
|
|
+### Component-based System Design
|
|
|
+
|
|
|
```clojure
|
|
|
(defn my-thing [main-ui] [:div "regular reagent component"])
|
|
|
|
|
@@ -634,4 +510,135 @@ Idea:
|
|
|
; keep kafka consumers in sync in a decoupled way :)
|
|
|
(add-watch subscriptions :update-consumers update-consumers)
|
|
|
|
|
|
-```
|
|
|
+```
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Aggregate Objects
|
|
|
+
|
|
|
+After two years of clojure and data stream programming, one begins to think of all objects as aggregates of the
|
|
|
+events that it they have experienced. What is a key/value store? A sequence of keys and values. How can we build one?
|
|
|
+
|
|
|
+1. `(write {:key "harlan.name.first" :value "Harlan"})`, we'll use `[harlan.name.first Harlan]` for brevity.
|
|
|
+2. `[harlan.name.last Iverson]`
|
|
|
+3. `[harlan.job Hair Stylist]`
|
|
|
+4. `[alice.name.first Alice]`
|
|
|
+5. `[alice.name.last Simply]`
|
|
|
+6. `[alice.job Programmer]`
|
|
|
+
|
|
|
+```
|
|
|
+{"harlan.name.first" "Harlan"
|
|
|
+ "harlan.name.last" "Iverson"
|
|
|
+ "harlan.job" "Hair Stylist"
|
|
|
+ "alice.name.first" "Alice"
|
|
|
+ "alice.name.last" "Simply"
|
|
|
+ "alice.job" "Programmer"}
|
|
|
+```
|
|
|
+
|
|
|
+7. `[harlan.job Programmer]`
|
|
|
+
|
|
|
+```
|
|
|
+{"harlan.name.first" "Harlan"
|
|
|
+ "harlan.name.last" "Iverson"
|
|
|
+ "alice.name.first" "Alice"
|
|
|
+ "alice.name.last" "Simply"
|
|
|
+ "alice.job" "Programmer"
|
|
|
+ "harlan.job" "Programmer"}
|
|
|
+```
|
|
|
+
|
|
|
+The rule here is to have one slot per key that can hold a value, and keep the latest one we receive; remember,
|
|
|
+these are write events. Thankfully Computer Science gave us the Dictionary data structure that we can leverage:
|
|
|
+
|
|
|
+```clojure
|
|
|
+(def assoc clojure.core/assoc) ; not needed, just for example. works on dictionaries, ka maps.
|
|
|
+
|
|
|
+(defn kv-store []
|
|
|
+ (let [store (atom {})]
|
|
|
+ (fn [event] (swap! store assoc (:key event) (:value event))))
|
|
|
+```
|
|
|
+
|
|
|
+We can image more interesting objects, like summation or moving average. I am using objects more in the logic sense than
|
|
|
+the OOP sense.
|
|
|
+
|
|
|
+```clojure
|
|
|
+(defn- inc [value] (+ 1 (or value 0)))
|
|
|
+
|
|
|
+(defn sum []
|
|
|
+ (let [value (atom 0)]
|
|
|
+ (fn [event] (swap! value inc)))
|
|
|
+```
|
|
|
+
|
|
|
+```clojure
|
|
|
+(defn- cma-pair [[i avg] value]
|
|
|
+ (let [new-i (+ 1 i)
|
|
|
+ new-avg (/ (+ avg value) new-i)]
|
|
|
+ [new-i new-avg]))
|
|
|
+
|
|
|
+(defn cma []
|
|
|
+ (let [state (atom [0 0])]
|
|
|
+ (fn [event] (second (swap! state update cma-pair (:value event))))))
|
|
|
+```
|
|
|
+
|
|
|
+
|
|
|
+These are foundational events. Let's try something more advanced:
|
|
|
+
|
|
|
+
|
|
|
+```clojure
|
|
|
+(defn cma []
|
|
|
+ (let [state (atom [0 0])]
|
|
|
+ (fn [event] (second (swap! state update cma-pair (:value event))))))
|
|
|
+```
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+Stream Middleware.
|
|
|
+
|
|
|
+https://github.com/pyr/riemann-kafka
|
|
|
+
|
|
|
+Riemann + Kafka...
|
|
|
+
|
|
|
+
|
|
|
+http://allthingshadoop.com/2014/04/18/metrics-kafka/
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+These aggregate objects can be composed using Clojure's inbuilt `(comp)` function.
|
|
|
+
|
|
|
+```
|
|
|
+;
|
|
|
+
|
|
|
+(defrecord EventsByCountry [users number-of-users]
|
|
|
+ (init []
|
|
|
+ {:users (atom (or users (kv-store)))
|
|
|
+ :number-of-users (atom (or number-of-users (count)))})
|
|
|
+ (process [forward events event]
|
|
|
+ (swap! )))
|
|
|
+
|
|
|
+(let [users (kv-store)
|
|
|
+ number-of-users (count)]
|
|
|
+ (do (users event)
|
|
|
+ (number-of-users event)))
|
|
|
+```
|
|
|
+
|
|
|
+
|
|
|
+They can then be placed in pipelines as xforms on core.async channels.
|
|
|
+
|
|
|
+```
|
|
|
+;
|
|
|
+
|
|
|
+(def input (chan))
|
|
|
+
|
|
|
+(def
|
|
|
+
|
|
|
+
|
|
|
+```
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+## Thanks
|
|
|
+
|
|
|
+Thanks to all the authors and contributors of the projects used and literature referenced.
|