voicerepublic_dev

Welcome to VoiceRepublic

Integration , Master

Platforms

• Live: http://voicerepublic.com

Dependencies

Ruby

VR is being developed on Ruby 2.1.2. See .ruby-version.

The use of rbenv and ruby-build instead of RVM is highly recommended.

Upgrade with rbenv

(cd ~/.rbenv && git pull)
(cd ~/.rbenv/plugins/ruby-build && git pull)
rbenv install 2.1.2
gem install bundler

Debian Packages

• postgresql-contrib-9.1
• libpcre++-dev
• libav-tools
• sox
• vorbis-tools
• libreadline-dev
• libpq-dev

make sure you install those before you proceed to the next point, since some (-dev) packages are needed in the following steps.

Install rbenv on zsh

git clone https://github.com/sstephenson/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(rbenv init -)"' >> ~/.zshrc
. ~/.zshrc
git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build
rbenv install `cat .ruby-version`

rbenv will install shim executables that understand rbenv settings. However you will face the problem, that your shell remembers the location of executables you have allready used. So before you use an executable that should be aware of rbenv you'll either need to launch a new shell or delete your shell's idea of where the executable is with:

hash -r some_executable

This is namely the case with bundle and cap.

Now install bundler and capistrano versions for the chosen ruby version:

gem install bundler
gem install capistrano

Now start a new shell or issue

hash -r bundle

Use spring

To make you shell find the springified binstubs, put this in your shell's config (e.g. .zshrc or .bashrc)

export PATH=./bin:$PATH

Setup

bundle
bin/update-local-settings

cp config/database.yml.example \
   config/database.yml             # use an appropriate db config

# edit template files

Database

rake db:setup                      # this requires an internet connection, also
                                   # [also see note below]
rake db:migrate                    # also requires an internet connection

Build docker images for icebox & worker (this will take a while!)

lib/icebox/build.sh
lib/audio_worker/build.sh

Optionally sync data from live (this will take even longer!)

rake db:sync:live
rake db:sync:live_images

Above the step rake db:setup probably fails, if the connecting user in database.yml doesn't have 'superuser' rights when connecting to the psql server. So you might want to replace that step by:

$ sudo su
# su - postgres
$ psql
postgres=# create database vr_development owner your_db_user;

New Search

Make sure postgresql-contrib-9.1 is installed.

rake pg_search:multisearch:rebuild\[Talk\]
rake pg_search:multisearch:rebuild\[Venue\]
rake pg_search:multisearch:rebuild\[User\]

Create Postgres Extensions

Sidenote: This is also automatically done by rake db:create.

# su - postgres
$ psql vr_development
vr_development=# CREATE EXTENSION pg_trgm;
CREATE EXTENSION
vr_development=# CREATE EXTENSION unaccent;
CREATE EXTENSION

Repeat for vr_test

Run App

voicerepublic_dev

• rails s -b 0.0.0.0
• rackup -E production faye.ru -o 0.0.0.0
• lib/flux_capacitor.rb run
• rake jobs:work
• cd lib/vrng && rlwrap lein figwheel
• docker run rabbitmq

or optionally voicerepublic_dev with tmux

• sudo apt-get install tmux
• bin/start_everything_in_tmux

voicerepublic_backoffice

• rails s -p 3001

Specs

Rails

Setup: Install phantomjs (globaly)

sudo npm install -g phantomjs

Run all specs

rspec spec

If you want to skip specs tagged as slow or to run in chrome, add this to your .rspec file

--tag ~@driver:chrome
--tag ~@slow

To run specs tagged to run in chrome

rspec --tag @driver:chrome

To run specs tagged as slow

rspec --tag @slow

Angular

Setup: Install karma & coffee-script

sudo npm install -g karma
sudo npm install -g karma-ng-scenario
sudo npm install -g coffee-script

Run Jasmine specs for Angular with Karma

karma start spec/javascripts/livepage.conf.js.coffee

Run Audio Strategies

See fidelity for details.

Example

Pull complete audio data for a given talk from s3

s3cmd sync s3://vr-live-media/vr-1799 .

Sometimes it is a good idea to delete the journal

rm vr-1799/1799.journal

Run fidelity on it

fidelity run vr-1799/manifest-1799.yml

To update fidelity, run

bundle update --source fidelity

Documentation

For general platform and development documentation please refer to the GitHub wiki pages.

Build diagrams

railroady -M | dot -Tsvg > doc/models_complete.svg
railroady -b -M | dot -Tsvg > doc/models_brief.svg
railroady -C | dot -Tsvg > doc/controllers_complete.svg
railroady -b -C | dot -Tsvg > doc/controllers_brief.svg

mscgen -T svg -o doc/streaming.svg doc/streaming.msc
mscgen -T png -o doc/streaming.png doc/streaming.msc

dot -Tsvg doc/fsm_venue.dot > doc/fsm_venue.svg
dot -Tpng doc/fsm_venue.dot > doc/fsm_venue.png

dot -Tsvg doc/fsm_device.dot > doc/fsm_device.svg
dot -Tpng doc/fsm_device.dot > doc/fsm_device.png

Working with Settings/Config

Config entries are compiled from:

config/settings.yml
config/settings/#{environment}.yml
config/environments/#{environment}.yml

config/settings.local.yml
config/settings/#{environment}.local.yml
config/environments/#{environment}.local.yml

Settings defined in files that are lower in the list override settings higher.

IMPORTANT! We break we this schema of reading config files when it comes to microservices. The services (in app/services) rarely boot the whole rails stack (this keeps the memory footprint low). As a side effect they don't have access to the whole settings. Only settings in the settings.local.yml will be available to microservices.

A note on emails

These files...

public/images/email_header.png
public/images/email_header.svg
public/images/icons/facebook.png
public/images/icons/link.png
public/images/icons/twitter.png

(except the svg) are linked to from...

app/views/layouts/mailer.html.erb

It makes since to put these NOT in app/assets/images, otherwise the MUAs will get 404s on emails that were sent from older releases.

Deploy

Deploying requires the presence of the ssh agent (because credentials will be forwarded to the target machine). So make sure, you have done a:

ssh-add

before you continue with:

cap staging deploy

Deploy a specific branch to staging, e.g.

REVISION=feature/65463494/subscribe_podcast cap staging deploy

HOWTO Manual Acceptance Tests (UAT)

If you don't have it already you realy should install tree.

% sudo apt-get install tree

To stream locally, build the docker container:

docker build -t branch14/icecast2 lib/icebox

Open two browser windows (with different sessions, one should be the host) and direct them to show the same talk. In a terminal start vrwatch. Attention: This will delete all previous recordings! (Never ever run this in production!)

% bin/vr_delete_and_watch

On a rails console, pick the talk you want to work with (e.g. id 42) and reset it to start soon, there is a nifty helper for that.

t = Talk.find(42); nil
t.reload.make_it_start_soon!

This will also trigger a reload of the browsers. It will record a couple of seconds pretalk. You should see the file size in 'watch' console go up. After the time to start is up it will switch to live mode automatically. With your second browser you should hear yourself now. If you don't have a headset plugged in this will very likely create a feedback loop. (Reload the host at least once if you want to have multiple files. These will show up on the 'watch' console.)

Click End Talk to end the talk. On the rails console kick off post processing (in User Acceptance Test mode) with:

t.reload.send(:postprocess!, true)

Views will change. A journal file shows up in the 'watch' console. When running with parameter true a debugger statement will hold before each Audio::Strategy to inspect the precondition and outcome in vrwatch. Type c and Enter to continue to the next strategy. It will output the shell-out-commands prefixed with CmdRunner>, any errors and in red the next strategy to run. (At this point you can run the shell-out-commands under local/recordings to play with them and tweak stuff.)

After the last strategy post processing will move the files from recordings to archive resp. archive_raw. It'll also create symlinks to access these files via public/system/audio.

At that point the history of you rails console will be dead -- no clue why. But you can simply quit the console and restart it to get it back.

Restart vrwatch to remove the artifacts and start over.

Troubleshooting Process/Monit

Start processes manually

$ current
$ be bin/init/private_pub start
...
$ be bin/init/localeapp start
...

Restart Monit to make it reinitialize the flags

# /etc/init.d/monit restart
...

Audio Cheat Sheet

For experimenting you might want to...

sudo apt-get install libsox-fmt-mp3

get duration

soxi -D file.wav

convert wav to flv

avconv -y -i file.wav -acodec libspeex -ar 16k -ac 1 file.flv

Make the best out of files which have silence at the beginning

sox --norm 90.mp3 90.wav
sox 90.wav 90-vad.wav vad

convert x.wav to x.ogg

oggenc x.wav

convert x.wav to x.mp3

avconv -y -i x.wav x.mp3

convert x.wav to x.m4a

avconv -y -i x.wav -b:a 64k -strict experimental x.m4a

resample wav to 44.1k and 2 channels

sox -c 2 vrs.wav lib/audio/files/vr_stop.wav rate -L 44.1k

collect stream info of multiple flv files

find ./ -name \*.flv -exec avconv -i {} \; 2>&1 | grep Stream

Get medadata of audio file

mediainfo -f 

Rails Console Cheat Sheet

Set a penalty on a user

 User.find_by(slug: 'back2us-radio').set_penalty!(0.5)

List popular with position, rank, penalty, id, and title

Talk.popular.limit(15).each_with_index do |t, i|
  puts [i+1, t.popularity, t.penalty, t.id, t.title]*"\t"
end; nil

Debug Postprocessing

id = 3322
Talk.find(id).update_attribute(:state, 'postlive')
Delayed::Job.enqueue(Postprocess.new(id: id), queue: 'audio')

Feature three randomly selected talks since yesterday

Talk.order('RANDOM()').limit(3).each do |t|
  t.update_attribute :featured_from, 1.day.ago
end

Reload browser session of all attendenees of talk 737

PrivatePub.publish_to '/t737/public', event: 'Reload'

Send Message as JavaScript Popup

PrivatePub.publish_to '/t1857/u1462094', { exec: 'alert("Hallo, hab gerade reingehoert, ich erklaere dir gerne wie du die Soundqualitaet merklich verbessern kannst. Meld dich dazu mal ueber unser Feeback Tool unten rechts. Gruss phil")' }

Enqueue all archived talks for processing

Talk.archived.order('play_count DESC').each do |talk|
  puts talk.id
  if talk.recording_override.blank?
    Delayed::Job.enqueue(Reprocess.new(talk.id), queue: 'prio')
  else
    Delayed::Job.enqueue(ProcessOverride.new(talk.id), queue: 'prio')