created HLS streaming container that proxies static container

.dockerignore

@@ -0,0 +1,2 @@
+.git
+

Dockerfile.streaming

@@ -0,0 +1,9 @@
+FROM nginx-rtmp
+
+# NOTE need to clone from github.
+# can be changed from jessie to xenial build-pack
+
+COPY nginx.conf /etc/nginx/nginx.conf
+
+WORKDIR /tmp/http
+ADD app/resources/public /tmp/http

README.md

@@ -1,9 +1,98 @@
-The site is in the apps directory. Based on cryogen.
+# iSpooge Live
 
-Built with `./scripts/build.sh`, started with `./scripts/start.sh` and will stay running after reboot... stopped with `./scripts/stop.sh`.
+A system for video websites and apps.
 
-The running service will rebuild changes, although it takes about 2 minutes on the RasPi and makes content unavailabile.
+No centralization unless you want there to be.
 
-In order to address this, we can create a snapshot with `./scripts/build-static.sh` and run that using `./scripts/start-static.sh` (heh) and `./scripts/stop-static.sh`.
+Syndicate in common formats. Sites with the POSSE.
+
+### Hosting
+
+The site is built from the ground up to be static with progressive enhancement
+on the client. It should be able to run offline, and also be enhanced by
+the latest and greatest cloud services. Video can be hosted as an original file,
+or transcoded for adaptive web and mobile playback. 
+
+It was built on a RasPi to be as constrained an accessible as possible, and to
+demononstrate the Tiny DataCenter project in action. It could be ported to run
+on any static web host; current deployment model is a simple Docker container
+that serves HTTP and static content.
+
+### Social Discovery
+
+It's 2018... people can discover your website via word of mouth, ping of text,
+and RSS aggregators.
+
+Each site has `/feed.xml` which is an RSS feed of the latest items. In the 
+`config.edn` file for the project, one can simply add tags to a list like
+`:rss-filters ["ispooge" "video"]` and then `/ispooge.xml` and `/video.xml`
+will be exposed with those items filtered.
+
+### Feedback
+
+Out of the box, feedback is solicited via Email and Phone number. Those enable
+a person to get started right away, without setting up IT systems or being 
+concerned with moderation. Many YouTube Live and Twitch channels are based around
+live feedback, so that could very easily become a priority.
+
+
+### Scalability
+
+Marketing can happen as normal. Being that the mechanics of decentralization are 
+different, the marketing channels and metrics will be different. 
+
+The IT scales by separating read and write paths intelligently, with read cache
+tunable at several levels, and scaling out of the box via CDN.
+
+
+## Usage
+
+Scripts are mostly named with verbs and nouns. Eg. if you `scripts/start-A.sh` there will be
+a `scripts/stop-A.sh`.
+
+From a high level, the site needs to be build with Cryogen before anything can
+be served. Once it's built, it needs to go into a static container. Around that
+static container goes the streaming container... in techno babel, the streaming
+container proxies to the static container.
+
+Media is served from the static container. Dev is currently possible without starting
+the streaming container, but it's so cheap you should. This will very likely soon
+change as more rules are added to the `nginx.conf` file.
+
+Note that we've not taken care to run multiple sites on one box yet... in theory
+just a few changes need to be made around mapping resources. It should be possible
+without much trouble.
+
+The example only uses the 360p stream, but you can extrapolate the others. The 
+config is set to take any number; by convention I end it with `_size`. Adaptive
+playback is set for 360, 720, 240 in `nginx.conf`.
+
+### Building
+
+* `scripts/start-build.sh` - starts the Cryogen builder process
+
+### Serving
+
+* `scripts/build-static.sh` - builds a snapshot from what Cryogen has built
+* `scripts/start-static.sh` - starts the snapshot that was built
+
+* `scripts/build-streaming.sh` - builds the live video streaming server
+* `scripts/start-streaming.sh` - starts the live streamig server we just built
+
+### Accessing
+
+Connect to `http://$HOST:33936/` and browse to `Live` on the menu bar, or go direct
+to `http://$HOST:33936/live-360p.html`.
+
+### Publishing
+
+Publish an RTMP stream such as OBS or a webcam to `rtmp://$HOST:33935/ispoogedaily_local/ispoogedaily_360`.
+
+
+### Syndicating
+
+Copy the RTMP stream you wish to syndicate upstream... usually the highest quality
+variant:
+
+* `ffmpeg -i rtmp://$HOST:33935/ispoogedaily_local/ispoogedaily_720 -c copy -f flv rtmp://_____`
 
-The static service uses the same Python module that we use for the v1 service.

nginx.conf

@@ -0,0 +1,226 @@
+# wraps around the static image to deliver HLS video streaming
+
+http {
+    upstream ispooge_static {
+        server ispooge-static:9090;
+    }
+
+    types {
+        application/vnd.apple.mpegurl m3u8;
+        video/mp2t ts;
+    }
+
+    server {
+        listen      1936;
+        listen [::]:1936 ipv6only=on;
+
+        location /hls/ {
+          rewrite  ^/hls/(.*) /$1 break;
+
+          # Disable cache
+          #add_header Cache-Control no-cache;
+
+          # CORS setup
+          add_header 'Access-Control-Allow-Origin' '*' always;
+          add_header 'Access-Control-Expose-Headers' 'Content-Length';
+
+          # allow CORS preflight requests
+          if ($request_method = 'OPTIONS') {
+              add_header 'Access-Control-Allow-Origin' '*';
+              add_header 'Access-Control-Max-Age' 0;
+              add_header 'Content-Type' 'text/plain charset=UTF-8';
+              add_header 'Content-Length' 0;
+              return 204;
+          }
+
+          root /tmp/hls/ispooge.com/;
+        }
+
+        # This URL provides RTMP statistics in XML
+        location /hls/stat {
+            rtmp_stat all;
+
+            # Use this stylesheet to view XML as web page
+            # in browser
+            rtmp_stat_stylesheet stat.xsl;
+        }
+
+        location / {
+          proxy_pass      http://ispooge_static/;
+        }
+    }
+}
+
+worker_processes auto;
+rtmp_auto_push on;
+events {}
+rtmp {
+    server {
+        listen 1935;
+        listen [::]:1935 ipv6only=on;
+
+        chunk_size 4096;
+
+        application participant {
+            live on;
+            record off;
+
+            allow publish 192.168.1.0/24;
+            deny publish all;
+        }
+
+
+
+        # monitor rtmp
+        application record {
+            live on;
+
+            allow publish 127.0.0.0/24;
+            deny publish all;
+            deny play all;
+
+            record all;
+            record_path /tmp/rec;
+            record_suffix _%Y-%m-%d_%H-%M-%S.flv;
+            record_lock on;
+        }
+
+
+        application hls {
+            live on;
+
+            allow publish 127.0.0.0/8;
+            allow publish 192.168.1.0/24;
+            deny publish all;
+            allow play all;
+
+#            pull rtmp://127.0.0.1:1935/ispoogedaily/ispoogedaily name=ispooge_com_hls live=1 static=1;
+
+            hls on;
+            hls_path /tmp/hls/ispooge.com;
+            #hls_base_url http://192.168.1.8:33080/hls/;
+            hls_fragment 500ms;
+            hls_playlist_length 3s;
+            hls_fragment_naming_granularity 500;
+
+            hls_continuous on;
+            hls_nested on;
+            hls_cleanup off;
+            hls_fragment_naming system;
+            hls_fragment_slicing aligned;
+            hls_type live;
+
+            hls_variant _720  BANDWIDTH=6491456 RESOLUTIO=1280x720;
+            hls_variant _360 BANDWIDTH=3345728 RESOLUTION=640x360;
+            hls_variant _240 BANDWIDTH=1248576 RESOLUTION=416x240;
+        }
+
+        # push to monitoring, push to recording
+
+
+        application  ispoogedaily {
+            live on;
+            record off;
+
+            allow publish 127.0.0.0/8;
+            deny publish all;
+
+            allow play all;
+        }
+
+        application ispoogedaily_local {
+            live on;
+
+            allow publish all;
+            allow play all;
+
+            push rtmp://127.0.0.1:1935 app=ispoogedaily live=1;
+
+
+            #push rtmp://127.0.0.1:1935/record live=1;
+
+
+
+            push rtmp://127.0.0.1 app=hls live=1;
+        }
+
+
+        application ispoogedaily_source_720 {
+            live on;
+
+            allow publish all;
+            allow play 127.0.0.0/8;
+            allow play all;
+
+            #push rtmp://127.0.0.1:1935 app=ispoogedaily live=1;
+
+
+            #push rtmp://127.0.0.1:1935/record live=1;
+
+            #exec /ffmpeg -hide_banner -y -i rtmp://localhost/$app -c:a copy -c:v h264 -profile:v main -sc_threshold 0 -g 50 -keyint_min 50 -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease -b:v 6000k -maxrate 6420k -bufsize 9000k -f flv rtmp://localhost/ispoogedaily_local/ispoogedaily_720;
+        }
+
+
+        application ispoogedaily_source_360 {
+            live on;
+
+            allow publish all;
+            allow play 127.0.0.0/8;
+            allow play all;
+            deny play all;
+
+            #push rtmp://127.0.0.1:1935 app=ispoogedaily live=1;
+
+
+            #push rtmp://127.0.0.1:1935/record live=1;
+
+            #exec /ffmpeg -hide_banner -y -i rtmp://localhost/$app -c:a copy -c:v h264 -profile:v main -sc_threshold 0 -g 50 -keyint_min 50 -vf scale=w=640:h=360:force_original_aspect_ratio=decrease -b:v 2800k -maxrate 3200k -bufsize 9000k -f flv rtmp://localhost/ispoogedaily_local/ispoogedaily_360;
+        }
+
+
+        application ispoogedaily_source_240 {
+            live on;
+
+            allow publish all;
+            allow play 127.0.0.0/8;
+            allow play all;
+
+            #push rtmp://127.0.0.1:1935 app=ispoogedaily live=1;
+
+
+            #push rtmp://127.0.0.1:1935/record live=1;
+
+            #exec /ffmpeg -hide_banner -y -i rtmp://localhost/$app -c:a copy -c:v h264 -profile:v main -sc_threshold 0 -g 50 -keyint_min 50 -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease -b:v 6000k -maxrate 6420k -bufsize 9000k -f flv rtmp://localhost/ispoogedaily_local/ispoogedaily_240;
+        }
+
+
+        # push to upstream services, including ispooge media
+        application  ispoogedaily_live {
+            live on;
+            #play_restart on;
+            drop_idle_publisher 15s;
+            #publish_notify on;
+            wait_key on;
+            wait_video on;
+            # interleave on;
+            #meta copy;
+
+            allow publish all;
+
+            deny publish all;
+#            deny play all;
+
+
+            # YT LIVE
+            push rtmp://a.rtmp.youtube.com app=live2 playpath=YT_KEY live=1;
+
+            # Twitch
+            push rtmp://live-sjc.twitch.tv app=app playpath=TWITCH_KEY live=1;
+
+        }
+
+
+
+
+    }
+}

scripts/build-streaming.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+source scripts/env.sh
+
+docker build -t $IMAGE_STREAMING -f Dockerfile.streaming .

scripts/env.sh

@@ -10,6 +10,9 @@ NAME=ispooge.com-build
 DEV_PORT=33000
 STATIC_PORT=33080
 
+RTMP_PORT=33935
+HTTP_PORT=33936
+
 
 DOMAIN=ispooge.com
 HOSTS=ispooge.com
@@ -18,3 +21,7 @@ HOSTS=ispooge.com
 NAME_STATIC=ispooge.com-static
 IMAGE_STATIC=docker-registry.local:5000/ispooge.com/ispooge-static:25
 IMAGE_STATIC_LOCAL=ispooge-static
+
+NAME_STREAMING=ispooge.com-streaming
+IMAGE_STREAMING=docker-registry.local:5000/ispooge.com/ispooge-streaming:1
+IMAGE_STREAMING_LOCAL=ispooge-streaming

scripts/start-static.sh

@@ -5,7 +5,7 @@ source scripts/env.sh
 
 docker run -d --restart always  \
   --name $NAME_STATIC \
+  -h ispooge-static \
   -v $ISPOOGE_MEDIA_PATH:/tmp/http/media:ro \
-  -v /tmp/hls/ispooge.com:/tmp/http/hls:ro \
   -p $STATIC_PORT:9090 \
   $IMAGE_STATIC

scripts/start-streaming.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+
+source scripts/env.sh
+
+docker run -d --restart unless-stopped  \
+  --name $NAME_STREAMING \
+  -h ispooge-streaming \
+  -v /tmp/hls:/tmp/hls \
+  -p 33935:1935 \
+  -p 33936:1936 \
+  --link $NAME_STATIC \
+  $IMAGE_STREAMING

scripts/stop-streaming.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+source scripts/env.sh
+
+docker rm -f $NAME_STREAMING