Thursday, January 09, 2014

HTTP Basic Authentication in AngularJS

Angular provides lots of support for REST services, so putting an Angular application in front of existing REST services looks easy at first sight. Indeed it is - as long as your REST services are exposed with no security at all.

Recently I changed the security in SimplePVR, which is my spare-time pet project. SimplePVR is a typical Angular application, where the back-end is a Sinatra application exposing both the JavaScript for the Angular application and a set of REST services used by the Angular application.

I used to secure everything in the back-end with HTTP Basic Authentication, with no special handling in the Angular application. This worked well, but had some problems:
  • The user was shown the browser's native log-in dialog, which is not pretty.
  • There was no obvious way to "log out", i.e. clear the HTTP Basic credentials.
  • Some browsers would show the log-in dialog for every single file fetched from the back-end, resulting in a very user-unfriendly experience.
Still, HTTP Basic Authentication is very nice for REST services, and adequately secure as long as you do HTTPS. I wanted to stick with HTTP Basic Authentication but give the user a better experience and a "log out" feature.

I thought that this had been done numerous times before, since Angular has great REST support, and HTTP Basic Authentication is widely used for REST services. It turned out to be much harder than I had anticipated, and hopefully this blog post will be useful to others. Join my odyssey...

About HTTP Basic Authentication

In case you don't know what HTTP Basic Authentication is all about, it works like this: The client starts by requesting a URL. The server looks at the requests, thinks "hey, I need to know who you are", and replies with HTTP status code 401 and the header "WWW-Authenticate" set to

Basic realm="My app"

The realm name can be whatever you want. A back-end can expose several different realms, in case different parts of the application requires different credentials.

The client receives this 401 response, see that it knows the authentication scheme set in the header ("Basic"), and in the case of a browser it will show a default log-in dialog to the user. The client will then retry the request with the "Authorization" header set to something like:

Basic dXNlcjpwYXNzd29yZAo=

where the gibberish part is the Base64-encoded user name and password separated by a colon - the above example is "user:password".

The server will decode the Base64 string and check the credentials. If the user name and password are correct, the real response will be returned. If not, the back-end will send another 401 response.

Since Base64 is just an encoding, not an encryption scheme, HTTP Basic Authentication in practice sends the user name and password in cleartext over the wire. Therefore, you really shouldn't use it for anything sensitive unless you're running HTTPS - just like any other log-in method where you send the user name and password directly to the server at any point.

It's a really nice and simple authentication scheme, and since all command-line utilities and all HTTP libraries out there support HTTP Basic Authentication in some way, it's perfect for REST services.

My initial plan

I had read a blog about handling authentication in AngularJS using promises. It's a great read, and it's a really beautiful use of promises: Register an HTTP interceptor which "catches" 401 errors, put these requests aside and let the application show a log-in dialog. Once the log-in dialog is completed, retry the requests with the new credentials. Since we're using promises, the code sending the original HTTP requests will never know or care about what's happening, even if the user takes hours filling in the log-in dialog.

So my plan was:
  • Secure only the REST services in the back-end, not the JavaScript code.
  • Plug in the aforementioned HTTP Authentication interceptor and handle the notifications sent by this by showing a nice Twitter Bootstrap log-in dialog.
  • Store the credentials somewhere in the client.
  • Implement a "log out" button which deletes these credentials.

Securing just the REST services

This was really easy: I created a Sinatra controller called SecuredController which secured controllers would inherit from. This would use Rack::Auth::Basic, like this:

  use Rack::Auth::Basic, 'SimplePVR' do |username, password|
    username == real_username && password == real_password
  end

The other controllers (those serving the Angular application) would be accessible without authentication.

Plugging in the HTTP Authentication interceptor

This was also quite easy. And I created an Angular directive which would simply show or hide a Twitter Bootstrap modal dialog (oh, so pretty!) according to the events sent by the interceptor:

directive('loginDialog', function() {
   return {
       templateUrl: '/app/templates/loginDialog.html',
       restrict: 'E',
       replace: true,
       controller: CredentialsController,
       link: function(scope, element, attributes, controller) {
           scope.$on('event:auth-loginRequired', function() {
               element.modal('show');
           });

           scope.$on('event:auth-loginConfirmed', function() {
               element.modal('hide');
               scope.credentials.password = '';
           });
       }
   } 
});

The form in the loginDialog.html template would, on submit, call a logIn function from the CredentialsController which would set the correct HTTP Basic Authentication header:

var encodedUserNameAndPassword = encodeBase64(userName + ':' + password);
$http.defaults.headers.common['Authorization'] = 'Basic ' + encodedUserNameAndPassword;

I found a nice Base64 encoding function here - by the way, that is one of the many blogs giving an incomplete solution to this whole very problem.

Storing the credentials

I'm not super proud of it, but I ended up storing the credentials in a cookie. I just stored the "encodedUserNameAndPassword" from above:

$cookieStore.put('basicCredentials', encodedUserNameAndPassword);

I did this because it was easier than checking whether the browser supports localStorage, and then storing it there - Angular comes with built-in Cookie support.

But if you think about it, this does not weaken the security of the system, since encodedUserNameAndPassword is sent in the Authorization header anyway.

One possible improvement would be to make the cookie secure when the back-end is running HTTPS, but I don't see an easy way to do this with Angular. I would definitely have done this for a more critical application.

That was supposed to be it

...but it didn't work. Before the HTTP Authentication interceptor could do anything with the HTTP 401 status code, the user was presented with this dreaded dialog (this is the Firefox version, but other versions are not prettier):

Surfing the web, there seems to be a lot of confusion about how to avoid this when accessing resources secured by HTTP Basic Authentication, but the conclusion is that it's not possible! The browser shows this dialog because it knows HTTP Basic Authentication, and only when the user enters invalid credentials will the calling JavaScript receive the response with status code 401.

So my pretty Twitter Bootstrap dialog would be shown only after the user had entered invalid credentials in the browser's native log-in dialog.

Damnation!

A work-around

Further surfing the web, I found somebody proposing simply changing the authentication method presented by the server when the request has the header "X-Requested-With: XMLHttpRequest", which e.g. JQuery automatically adds to all Ajax requests.

So instead of simply using Rack::Auth::Basic, I had to implement my own authentication in Sinatra, which acts as a normal HTTP Basic Authentication when not called by Ajax, and which presents something else than "Basic" for Ajax requests. I chose "xBasic", but as long as it's something that the browser does not recognise, all is fine.

Angular actually does not set the X-Requested-With header like JQuery does, so I had to insert this configuration line:

$httpProvider.defaults.headers.common['X-Requested-With'] = 'XMLHttpRequest';

It worked!

The REST calls from the Angular application would now actually fail without any native log-in dialog and let our application handle the error. My nice Twitter Bootstrap modal log-in dialog was shown, and the user could enter the credentials. After that, all requests would get the Authentication header set with the entered credentials.

All seemed well, but then I tried first entering wrong credentials, which would make the request retries fail, and then entering the correct credentials. Something weird happened: The new request retries still had the old credentials set in the Authorization header.

I scratched my head quite a lot - why didn't Angular see that I had changed the default headers with this line I showed you previously??

$http.defaults.headers.common['Authorization'] = 'Basic ' + encodedUserNameAndPassword;

Then it dawned on me: I had previously set the default value of the Authorization header with the wrong credentials, and all requests from then on would have this header set. When I was changing the default and the HTTP Authentication interceptor was retrying "old requests", these "old requests" had already got an Authorization header, and so Angular didn't care about the new default value.

So in the HTTP Authentication interceptor, right before the request retry, I had to insert this line of code to make room for the new default value:

delete config.headers['Authorization'];

You can see the altered interceptor service here.

It worked!

It's a pity that this requires changes in the server, as it means that you cannot just put an Angular application in front of an existing REST service without changing it.

But I was happy and thought I had got everything right. The Angular application was accessing the REST back-end perfectly, I had a log-out button, I had my pretty Twitter Bootstrap log-in dialog, the REST services were still directly accessible with HTTP Basic Authentication (which means that command-line tools and my XBMC plug-in worked with no changes).

But... just one more thing...

Sometimes the REST service would serve an image URL, and the URL would be put in a normal image tag. Now the browser was fetching the image from the secured REST service... and this guy was back:

I was in despair for some time - should I choose to expose the images without security, or should I ditch HTTP Basic Authentication altogether?

It occurred to me that I could implement something of a hack in my server-side authentication code: If the request does not specify HTTP Basic Authentication credentials in the standard header, it will look in the aforementioned cookie.

This further adds to what has to be changed in your REST service if you are planning "just" adding an Angular front-end, but all things considered it's still a very simple authentication scheme.

As for the back-end authentication code, here's the test, and here's the implementation.

Fixing a GUI lock-up

A final problem was the modal dialog: Hiding and showing it too quickly didn't correctly clean up the backdrop, resulting in a darker and darker, inaccessible page behind the modal dialog, even when the user eventually entered the right credentials. I ended up delaying the "show" part. Have a look at the code if you care.

Friday, July 06, 2012

MythTV 0.25, HDHomeRun, and a remote on an EEE Box (unfinished...)

Background

I've previously installed Mythbuntu and used it happily, but after upgrading to 0.25 (and Mythbuntu 12.04 LTS), I've had severe glitches when recording most channels. This is most unsatisfactory. So I decided to do a complete reinstall and see if the problem persisted. It did. However, I've documented everything I did, so that when I find out what's wrong, this will hopefully serve as a step-by-step guide for others.

But until then: If you have an EEE Box and an HDHomeRun, I cannot recommend you to install Mythbuntu 12.04. Use an earlier version, which has been working nicely for me.

By the way: I'm sorry that the formatting on this page is a bit messed up. I'm just not able to format blog posts on Blogger decently...

Installation


Create USB disk

Download the 64-bit edition of Mythbuntu. Create a USB disk as described in the official Ubuntu documentation (I have a Mac, so I followed this).


Boot

Put the USB disk in the EEE Box, boot the machine. Hold down F2 during boot to go to the BIOS menu. Go to the Boot tab, select "Hard Disk Drives", press enter. In the list of hard disk drives, you should see both the real hard disk and your USB disk. Re-arrange the list so that the USB disk is first, and your hard disk is second. Save and exit, and your machine should now boot from your USB disk.


Install

Choose "Install Mythbuntu" when the welcome screen appears. You can choose "Download updates while installing", and I guess you need to select "Install this third-party software". Click "Continue".

Since I had a previous Mythbuntu install, I had the chance of installing the new side by side. I chose the "Erase and reinstall" option.

As for disk partitioning, I like to have 20GB for the actual Mythbuntu install, 5GB swap space, and the rest as a separate partition for my recordings (mounted as "/data"). Choose what fits you the best. And remember to select the right "Device for boot loader installation" (the guide had chosen the USB disk, which is obviously wrong).

As for "Installation type", choose "Primary Backend w/Frontend".

For "Additional Services", I chose "VNC Service", "SSH Service", and "MythTV service". I deselected "Samba Service" and left "NFS Service" unselected. Do what you want here.

When asked about "Infrared Remotes", choose "USB & Serial Remote support via LIRC (Linux Infrared Remote Control", and choose "Windows Media Center-Transceivers/Remotes (all)", make sure that "Generate dynamic button mappings" is checked.

As "Graphics Drivers", select "NVIDIA Graphics", and disable TV-Out. That should finish off the installation.

After rebooting, Update Manager wants to update some packages. Let it. And restart your box.

Backend setup

Quit Mythfrontend, start MythTV Backend Setup.

General

Go to "General", and set your "Locale Settings". I've chosen "PAL" as "TV format", "PAL teletext" ad "VBI format", and "europe-west" as "Channel frequency table".

Storage paths

Before you start using MythTV for real, remember to adjust the paths in MythBackend that define where recordings, live-tv, etc. go. It's under "Storage Directories". If, like me, you've mounted most of the EeeBox harddisk under "/data", you probably want values such as "/data/mythtv/recordings", "/data/mythtv/livetv", etc.

You should also create these directories in the file system, and remember to give proper rights to the mythtv user. I've just set all these directories to the mythtv user, the mythtv group, and given all rights possible. Perhaps that's a bit too much, but I don't want to spend time fiddling with this.

Tuner

Create a new "Capture card", with Card type "HDHomeRun DTV tuner box". As "Available devices", choose the one with your serial number and ending in "-0". Create yet another "Capture card", now choose the one with your serial number and ending in "-1".

Create a new "Video source". What you choose here depends heavily on where you are, etc. I have an account at a Danish XMLTV provider which is not officially supported by MythTV, so I choose "No grabber" as "Listings grabber". As name, I have chosen "ontv".


Go to "Input connections" and connect your two HDHomeRun tuners to the new video source: Select one line at a time, and select your video source under "Video source".

Channel scan

First, you need to find out where your channels are situated. Run

hdhomerun_config 12106FA4 scan 0

(where 12106FA4 is your tuner's serial number.) This will show you at which frequences and modulations your channels are.

Go to "Channel Editor", and choose "Channel Scan". Choose "Full Scan (Tuned)" as "Scan type", and enter the various frequencies and modulations and scan for each.

Sound and picture quality


Sound via HDMI or TOSLINK

If you want sound through the HDMI cable, create the file /etc/asound.conf with the following contents:

pcm.!default {
  type hw
  card 0
  device 3
}

If instead you want sound through the optical digital out, specify device 1.

Start "alsamixer" from a terminal and unmute "S/PDIF 1" by clicking M on it.


Picture quality

Turn on VDPAU in the "Setup -> Video -> Playback", otherwise the EeeBox doesn't stand a chance when playing HD content.

Still, playback isn't always perfect, since the refresh rate on your TV doesn't necessarily match the refresh rate of the played material. To fix this, you need to do two things: First, go to "Setup -> Appearance", and on the "Video Mode Settings" page, check "Separate video modes for GUI and TV playback". Just choose 1920x1080 as the GUI and Video output resolution, and choose "Auto" as the rate. Don't change the values in the "Overrides for specific video sizes" section.

Secondly, open nvidia-settings and uncheck the "Force Full GPU Scaling" on one of the pages. Voila! Playback is now perfect! (At least on my setup :-) )

Importing XMLTV


First, set all the XMLTV IDs for your channels: Go to MythWeb, click the "MythTV" link in the upper left, and choose "Settings -> TV -> Channel Info".
Since I get my XMLTV from a specific URL, I have a mythfilldatabase.sh script which is quite simple:


#!/bin/bash


LOGFILE=/home/me/mythfilldatabase.log


source /home/me/.profile


mv /home/me/tvgrabbed.xmltv.1 /var/log/mythtv/tvgrabbed.xmltv.2
mv /home/me/tvgrabbed.xmltv /var/log/mythtv/tvgrabbed.xmltv.1


cd /home/me/
wget --output-document=/home/me/tvgrabbed.xmltv http://ontv.dk/xmltv/SECRET  
/usr/bin/mythfilldatabase --file --sourceid 1 --xmlfile /home/me/tvgrabbed.xmltv > ${LOGFILE} 2>&1

It just wgets the xmltv file, loads it, and stores the last two days of xmltv files. (Of course, replace "me" with your user name and SECRET with your ontv ID if you want to use the above.)



Then set up a Cron job: Run "crontab -e", define a line like:

00 06 * * * /home/me/mythfilldatabase.sh

Various settings

Securing MythWeb

Open "Mythbuntu Control Centre" (start it as root), go to the "Plugins" page, and enable password protection.

Start/end of recordings

It's nice to start recording e.g. 2 minutes before the official starting time, and end recording e.g. 5 minutes after the official ending time. In MythWeb, you can alter the setting "DefaultStartOffset" and "DefaultEndOffset". Both are specified in minutes, so I've set them to 2 and 5, respectively. This implies that the default values of the fields "Start Early" and "End Late" in the recording dialog page will be set to 2 and 5 minutes.

Current status

Well... some channels work fine, and others have glitches, just like some network packets get lost.

The problem is not network-related: I've tried wiring the EEE Box and the HDHomeRun directly to each others (setting "Link-Local Only" in "IPv4 Settings" in the network config GUI). MythTV found the HDHomeRun and could show the channels, but with exactly the same problems as before.

The problem is not related to the EEE Box: I can start hdhomerun_config_gui, scan for channels, and see all channels perfectly with VLC. No glitches. (And I can do the same with other machines on the network, naturally.)

You can see an example here. The first few seconds are not that bad, but then the artifacts start. For more action-packed sequences, the artifacts make the recordings unwatchable.

By the way, my problems look a lot like these, except that my problems don't seem to be network-related.

Saturday, October 08, 2011

NemID on Rails

NemID on Rails
For nylig har jeg set et tweet eller to fra folk der gerne vil kunne banke et NemID-login op på en hjemmeside i et Ruby on Rails-projekt. Derfor tænkte jeg at en lille guide til at få NemID-login op at køre i Ruby on Rails kunne være til nytte ude i verden.


Jeg vil holde mig til at få NemID til at fungere i JRuby. Skal man have det til at fungere i flere Ruby-implementationer, kræver det en del arbejde (mere herom nedenfor).


Jeg vil ikke forklare hvad NemID i bund og grund er for noget. Det har DanID gjort fint på deres egen side, https://www.nemid.nu/.

Der findes en såkaldt Tjenesteudbyderpakke med noget eksempel-kildekode til Java og .NET, så hvis man har mod på at forstå det, vil nedenstående måske virke trivielt.

Formatering
Tilsyneladende er Blogger et sindssygt dårligt sted at skrive kodeeksempler. Mange linjer bliver ombrudt så det bliver meningsforstyrrende, og HTML-tags i tekst er den meget forvirret over.

Undskyld.

Scope
NemID består faktisk af to løsninger: En til bankerne, og en til det offentlige. Vi vil fokusere på den offentlige løsning, da private firmaer kan få lov at gøre brug af denne løsning. Derfor er det sandsynligvis denne løsning du er interesseret i.

DanID har lavet nogle fine udkast til design af login-sider. I vores eksempel vil vi dog nøjes med en kedelig, hvid stil. Pointen med dette blog-indlæg er ikke at "style" en rigtig login-side, kun at få login til at fungere.

NemID giver mulighed både for login og signering af dokumenter. Vi holder os til login. Hvis du er interesseret i en tilsvarende guide til signering, så giv lyd i en kommentar nedenfor.

Desuden tilbyder DanID nogle services til, på baggrund af certifikatoplysningerne ifm. et login, at slå et tilhørende cpr-nummer op. Det kræver ekstra aftaler med DanID, og vi vil heller ikke komme ind på det.

Forberedelse
Før du kan følge skridtene her på siden til noget, er du nødt til at få dig en såkaldt tjenesteudbyder-aftale med DanID. Ud over at det giver mulighed for at benytte NemID på din egen hjemmeside, giver det dig adgang til DanID's testmiljøer. Og før du har adgang til disse testmiljøer, vil ikke engang test-projektet her på siden kunne køre hos dig, idet DanID bruger IP-blokering på testmiljøerne.

Når du indgår aftale med DanID, får du et certifikat registreret hos DanID. Dette certifikat skal bruges ifm. opsætning af applet'en, således at denne starter op med dit firmanavn.

DanID vil også fortælle dig hvordan du opretter testbrugere i testmiljøet.

OOAPI
OOAPI (Open Oces API) er et open source-library som findes både til Java og .NET. Det indeholder funktionalitet til at læse certifikater, lave gyldighedscheck på certifikater osv. Desuden kender dette library til de ekstra certifikat-felter som NemID benytter sig af.

Hvis man vil bruge NemID uden for JRuby eller IronRuby, er man nødt til at gen-implementere dele af OOAPI. Det er helt sikkert en hyggelig, fin opgave, men i dag holder vi os til blot at bruge OOAPI i Java-udgaven.

Selve OOAPI er blot en jar-fil, og den hentes fra DanID's hjemmeside for tjenesteudbydere. Denne side er dog ret svær at finde, så for nemheds skyld får du her et direkte link:

https://www.nets-danid.dk/produkter/for_tjenesteudbydere/nemid_tjenesteudbyder/nemid_tjenesteudbyder_support/tjenesteudbyderpakken/

Hent ooapi-1.81.2-with-dependencies.jar.

Tjenesteudbyder-eksemplet
Vi skal desuden, for at få vores legetøjseksempel til at fungere, have et keystore med et legetøjs-certifikat som i DanID's testmiljøer er registreret under tjenesteudbyderen "www.nemid.nu".

Hent derfor tuexample-source.zip fra ovenstående side og snup filen applet-parameter-signing-keystore-cvr30808460-uid1263281782319.jks.

Faktisk har du i tuexample-source.zip al information der skal til for at gennemføre alle scenarier med NemID: Login for privatpersoner, login for medarbejdere, signering osv. Og det er ikke alt for kompleks kode. Men det kan du altid vende tilbage til hvis du får behov for det.

Oprettelse af Rails-projektet
Først skal du sikre at du har installeret og bruger JRuby. Bruger du RVM, er det blot at skrive

rvm install jruby
rvm use jruby

fra kommandolinjen.

Lav en mappe hvori du har ooapi-1.81.2-with-dependencies.jar og applet-parameter-signing-keystore-cvr30808460-uid1263281782319.jks liggende. Kør dernæst følgende fra kommandolinjen:

export CLASSPATH=$JAVA_STUFF/:/$JAVA_STUFF/ooapi-1.81.2-with-dependencies.jar 

(...hvor altså $JAVA_STUFF skal pege på den nævnte mappe med de to filer...)

Så lad os oprette et Rails-projekt:

rails new NemIdOnRails
cd NemIdOnRails


Environments
BouncyCastle skal sættes som Security Provider i JVM'en, så følgende skal tilføjes config/application.rb:

# Registrer BouncyCastle som Security Provider
java.security.Security.addProvider org.bouncycastle.jce.provider.BouncyCastleProvider.new

OOAPI skal have at vide hvilke miljøer det skal stole på. Her bruger vi DanID's "ekstern test". Tilføj følgende i bunden af config/environments/development.rb:

org.openoces.ooapi.environment.Environments.setEnvironments(org.openoces.ooapi.environment.Environments::Environment.value_of('OCESII_DANID_ENV_EXTERNALTEST'))

Desuden skal vi have styr på hvor det førnævnte keystore ligger, hvordan vi får fat i indholdet, og hvilken URL vi bruger for at tilgå vores testmiljø. Så sæt yderligere følgende ind i config/environments/development.rb:


ENV['key_store_url'] = 'applet-parameter-signing-keystore-cvr30808460-uid1263281782319.jks'
ENV['key_store_password'] = 'Test1234'
ENV['key_store_alias'] = 'alias'
ENV['key_password'] = 'Test1234'
ENV['server_url_prefix'] = 'https://syst2ext.danid.dk'

Når du kører i produktion, vil du bruge et andet keystore, og sikkert også nogle andre passwords... og server_url_prefix får du fra DanID.


Controllers, views, routes...
Vi vil lave en meget, meget simpel applikation med to controllers: Sessions og SecretPages. Sessions bruges til login, og SecretPages beskytter sin ene side, således at man skal være logget ind for at se den.

Vi beslutter at man er logget ind hvis den aktuelle session har en "pid"-attribut. PID er et felt i et personligt NemID-certifikat som unikt identificerer en person. Altså lidt som et cpr-nummer. Man kan derfor oplagt bruge PID'en som nøgle i en brugerdatabase.

Så lad os generere de to controllers:



rails g controller Sessions
rails g controller SecretPages


Og så skal vi lige opdatere config/routes.rb lidt:

resources :sessions
match 'hemmelig', :to => 'secret_pages#index'

Visning af applet'en
Vores Sessions-controllers new-action skal vise applet'en. Der er noget arbejde både i controlleren og i vores view.

For at undgå "replay attacks" finder vi på en tilfældig streng som brugeren via sit login signerer, og som vi så efterfølgende, når login er foretaget, checker i det svar som applet'en giver. Det er alt hvad vi gør i vores controller:


def new
  session[:challenge] = SecureRandom.hex(10)
end


Selve view'et, app/views/sessions/new.html.erb, indeholder lidt kode, men er ikke vildt kompleks:


<%
signer = org.openoces.ooapi.web.Signer.new(ENV['key_store_url'], ENV['key_store_password'], ENV['key_store_alias'], ENV['key_password'])


generator = org.openoces.ooapi.web.OcesAppletElementGenerator.new signer
generator.server_url_prefix = ENV['server_url_prefix']


generator.challenge = session[:challenge]
generator.log_level = 'debug' # INFO/DEBUG/ERROR
%>

<h1>Log ind</h1>
<%=raw generator.generate_logon_applet_element sessions_path %>

Her bruger vi så alle de værdier vi har sat i development.rb tidligere. Desuden bruger vi OOAPI til at generere selve applet-tag'et til siden. I den sidste linje beder vi desuden applet'en om at POST'e tilbage til "sessions_path".

Prøv det! Kør "rails s" fra kommandolinjen, og gå ind på http://localhost:3000/sessions/new.

Check af svaret
Som sagt har vi bedt applet'en om at POST'e sit svar ind på "sessions_path", dvs. vi rammer vores create-metode i SessionsController.rb. Der skal heldigvis ikke så meget til at checke resultatet fra applet'en... lad os starte med følgende:

def create
  result = org.openoces.securitypackage.SignHandler.base64Decode(request[:result])
  if result == 'ok'
    handle_ok
  elsif result == 'cancel'
    redirect_to :cancelled
  else
    redirect_to :unknown_action
  end
end


Så det er ret nemt at se om brugeren rent faktisk ønskede at gennemføre et login, eller om vedkommende fortrød først. Men det er jo ikke nok at brugeren bare har trykket "OK" - vi forventer også et gyldigt certifikat, og oven i købet et personligt certifikat (i modsætning til fx et virksomhedscertifikat), og vi ønsker som tidligere nævnt at huske PID'en fra certifikatet:


def handle_ok
  certificate_and_status = get_certificate_and_status 
  if certificate_and_status.certificate_status() != org.openoces.ooapi.certificate.CertificateStatus.value_of('VALID')
    redirect_to :invalid_certificate
  elsif !is_poces(certificate_and_status)
    redirect_to :wrong_certificate_type
  else
    session[:pid] = certificate_and_status.certificate.pid
    redirect_to '/hemmelig'
  end
end

Også her er der et par løse ender, nemlig hvordan vi haler certifikatet og status ud af request'et, og hvordan vi checker at der er tale om et personligt certifikat. Først certifikatet og status:

def get_certificate_and_status
  signature, challenge, service_provider = request[:signature], session[:challenge], 'www.nemid.nu'
  org.openoces.securitypackage.LogonHandler.validateAndExtractCertificateAndStatus(org.openoces.securitypackage.SignHandler.base64Decode(signature), challenge, service_provider)
end

Så det hele ordnes af OOAPI. Bemærk brugen af 'www.nemid.nu'. Her vil du, når aftalen med DanID er på plads, indsætte det navn som DanID har registreret dig under. Desuden giver vi vores tilfældige streng fra tidligere med, så OOAPI checker at brugeren har signeret dette.

Det er nemt at checke at vi har fået fat i et personligt certifikat:

def is_poces(certificate_and_status)
  certificate_and_status.certificate.kind_of? org.openoces.ooapi.certificate.PocesCertificate
end

Men en lille ting mere... Det er jo applet'en der kalder tilbage til vores controller, og det sker jo så uden Rails' indbyggede CSRF-beskyttelse. Så ovenstående kode vil bare give en fejl når brugeren logger ind. Men da vi har vores tilfældige streng og checker denne, er vi sikre på at alt er sikkert alligevel. Så lad os slå CSRF-beskyttelse fra på vores create-metode ved at skrive følgende øverst i vores SessionsController:

protect_from_forgery :except => :create

Åh ja... og vi vil jo også gerne kunne logge ud igen, dvs. nedlægge vores session:

def destroy
  session[:pid] = nil
  redirect_to :action => :new
end

I ovenstående kode har vi et par ekstra sider vi redirigerer til når noget er gået galt. Det efterlades som en opgave til læseren at implementere dette :-)


Den hemmelige side
Tjah, der er vel ikke så meget at sige om SecretPagesController. Den beskytter sin index-metode:

def index

  redirect_to :controller => :sessions, :action => :new unless session[:pid]
end

I det tilhørende view (app/views/secret_pages/index.html.erb) vil vi gerne se vores PID, samt have mulighed for at logge ud igen:



<h1>Tillykke!</h1>

Det lykkedes dig at logge ind! Din PID er <%= session[:pid] %>.


<%= link_to 'Log ud igen', session_path(1), :method => :delete %>


Det var så det
Mere er der ikke i det. Men som sagt, for overhovedet at kunne starte på denne lille trin-for-trin-guide skal du have en aftale i stand med DanID først.

Som det sig hør og bør, så har jeg lavet et projekt på GitHub med al koden, så du kan se om jeg har snydt: https://github.com/olefriis/NemID-Rails-login

Held og lykke!

Monday, June 13, 2011

Programming with the Stars at GOTO Copenhagen 2011

At GOTO Copenhagen 2011, we had a track called Programming with the Stars. It gave me a unique chance watching five software professionals along with an audience member solve the same (simple) problem in different programming languages, using different programming paradigms. It would be a shame not to reflect a little on what I learned that day.

Overview
In every time slot, a speaker was asked to solve the Tennis Kata along with a volunteer from the audience. The big idea was to start discussions on how to write code, how to apply different programming paradigms, and to show off the different languages and techniques to the audience.

First, Ulf Wiger did the Erlang presentation. There was even media coverage on this presentation. Ulf was very relaxed and cool and showed a lot of nifty language features. Alongside his text editor, he had a console where he poked around in the running code, showing how it worked. The result can be seen here.

Then, Patrick Linskey did a solution in Java. The result can be seen here. Apart from doing a solution in Java, Patrick was keen to check code into Git along the way, and we had a unit test showing that the code worked. I ended up as his partner, since the Java audience was a bit shy. It was great fun.

Next up was Mark Seeman in C#. Mark is a hard-core TDD master, and his partner clearly had no TDD experience. This was an extremely entertaining learning experience to everybody, as Mark was struggling to keep his partner from writing code before test and keeping things simple. The result can be seen here.

Talking entertainment, Sam Aaron was up next for the Ruby session, and he brought with him a monome. He had programmed it so that when he ran the tests (or rather, RSpec specifications), it would flash the right colors and perform adequate sounds based on the test outcomes. The result can be seen here. Frank Vilhelmsen took some great pictures of the event.

Finishing off, Christophe Grand showed us how to do the solution in Clojure (with Sam Aaron as partner, in fact). Just as Ulf, Christophe tried out the solution in a console (the so-called REPL) while writing the code. The result can be seen here.

Programming styles
Of course it's unfair to use the results of the different time slots as representations of the programming styles used in the different programming camps, since the track was more about process and learning than the actual result. Unfair or not, though, let's dive in...

I've started to appreciate Erlang. I used to think that it could only be used for telecommunication hubs and other stuff that was irrelevant to me, but Ulf showed lots of nifty stuff. Pattern matching works well in Erlang, and record manipulation is concise. However, it still feels a bit old-fashioned to me, e.g. you cannot reference fields in records in other modules by name until you specifically load a record definition. Ulf explained that there was a reason for this (very loose coupling, so you can upgrade parts of your application).

The Java solution is different from all the other solutions in that it models players and a game, and we have a definition of a finished game, a winner and all. This is probably overkill for the simple Tennis Kata game, but is typical Java. Why do we Java programmers over-engineer like this? Java is very verbose, so we probably wouldn't have had the time to write all this code without a proper IDE.

The C# code ended up very simple, since Mark was struggling to get the process straight. He used AutoFixture as a testing tool, something I'd really like to try out soon. We didn't get to see much of it, but it seems like a nice testing productivity tool. My verbosity comments for Java mostly apply for C# as well.

Ruby... I love it. The code, the specs, it all looks and feels natural to me. Compared to the two functional language results, though, I do see that it's a little verbose. However, I think it's OK to be a little bit verbose, as long as the verbosity serves readability.

I have to admit that I still haven't learned to appreciate Clojure. The technology is really cool (memory transactions, persistent data structures, etc.), but the LISP style is a problem for me. The following Clojure code:

(defn result
  "Returns a string representation of the game."
  [[score-a score-b]]
  (cond 
    (= score-a score-b) (if (>= score-a 3) 
                          "Deuce"
                          (str (points score-a) " all"))
    (and (> score-a 3) (= score-a (inc score-b))) "Advantage A"
    (and (> score-b 3) (= score-b (inc score-a))) "Advantage B"
    (and (> score-a 3) (> score-a (inc score-b))) "Game A"
    (and (> score-b 3) (> score-b (inc score-a))) "Game B"
    :else (str (points score-a) " " (points score-b))))

can be written (almost) as concise in Java/C#/Ruby:

String result(int scoreA, int scoreB) {
  if (scoreA == scoreB) {
    return scoreA >= 3 ? "Deuce" : scoreA + " all";
  }
  if (scoreA > 3 && scoreA == scoreB + 1) return "Advantage A";
  if (scoreB > 3 && scoreB == scoreA + 1) return "Advantage B";
  if (scoreA > 3 && scoreA > scoreB + 1) return "Game A";
  if (scoreB > 3 && scoreB > scoreA + 1) return "Game B";
  return points(scoreA) + " " + points(scoreB);
}

I don't know... what you like the most probably depends on what you're used to.

To test or not to test?
Both the Java and C# examples were made strictly TDD, and I think it worked really well. Having the tests drive the design enables the audience to follow exactly what's going on. At the end of the session, everybody knew that the solution worked.

It seems like test-driven development is having a hard time in the functional language fragment. Just try looking up "testing" in Joe Armstrong's Erlang book, and you'll see what I mean. It's easy to start up a prompt and mess around with your code and obtain confidence that way, so perhaps that's all a functional programmer needs? I'm a bit curious about the effect this has on long-term projects with many team members, though.

The really interesting case was the Ruby session. Sam and his partner started poking around with no tests, just to get some basic design fleshed out. That worked well. However, they held back doing the tests for too long, mainly because Sam's partner and part of the audience didn't feel it was necessary to start testing. However, when finally they started doing RSpec tests (with the monome!), they went from having no clear idea about their state to knowing exactly what worked and what needed to be done next. An amazing transformation!

Different cultures
Whoever turns up for a specific session depends on lots of things on a conference, but it was interesting to reflect on the audience for the different sessions.

The C# audience seemed pretty young and enthusiastic. The Java audience was the biggest, but also the most shy: nobody volunteered as Patrick's partner, and there were very few comments from the audience during the session. The Clojure session was packed with other speakers from the conference.

In conclusion
Being part of the organization team, I was supposed to stay at the track the whole day, and that turned out to be very rewarding. Seeing people with experience in the 5 different languages and paradigms solve the same problem gave a much better feeling of the languages than reading 5 different introductory articles.

Sunday, January 23, 2011

So, TDD is hard? Start decoupling!

As a lot of other TDD people, I try to push TDD onto my colleagues. It seldom works, and I think that's a shame. Therefore, lately I've begun to think about why a lot of other people find TDD painful, unproductive, and plain irritating when my own experience is the direct opposite.

I'd like to start a discussion about doing TDD and other agile practices in the real world, with real-world code examples, and this is the first blog post in this series.

The thing I want to point out in this post is that most of the time, people holding a grudge against TDD don't practice enough decoupling. Let's take an example of some code (practically) from the real world that I stumbled upon the other day. I understand those who wouldn't like to test this.

public ModelAndView createUser(long applicationId, String comment) {
  Application application = applicationDao.get(applicationId);
  application(OrderState.GRANTED);
  if (comment != null && !comment.isEmpty()) {
    application.setProcessingComment(comment);
  }
  applicationDao.save(application);
  Address address = application.getContactAddress();
  SocialSecurityNumber socialSecurityNumber = order.getSocialSecurityNumber();
  if (userDao.findUser(socialSecurityNumber) == null) {
    userDao.createUser(socialSecurityNumber, address);
  }
  if (application.isAdministrator()) {
    HashSet roles = new HashSet<role>();
    roles.add(new AdministratorRole(timeService.getTime(), null));
    roleDao.addRoles(socialSecurityNumber, roles);
  }
  return new ModelAndView("users/create", "message", "User created");
}

So, how would we test this? Well, we need to create a mock for an ApplicationDao, UserDao, and RoleDao. And we need to mock these method calls:

applicationDao.get(...), returning an Application
applicationDao.save(...)
userDao.findUser(...), returning a User
userDao.createUser(...)
timeService.getTime(), returing a Date
roleDao.addRoles(...)

But even worse, our logic needs to know a lot about different objects:

  • How to change an application so it gets to the "granted" state.
  • How to make sure that a user exists in the database.
  • How to promote a user to administrator.
  • How to construct an AdministratorRole.

This directly affects the cyclomatic complexity, so we need a huge amount of test cases to cover all possible paths through the code.

I understand why people find TDD frustrating when writing this kind of code.

But let's refactor it just a little bit:

public ModelAndView createUser(long applicationId, String comment) {
  Application application = applicationDao.get(applicationId);
  applicationDao.grant(application, comment);
  Address address = application.getContactAddress();
  SocialSecurityNumber socialSecurityNumber = order.getSocialSecurityNumber();
  userDao.ensureUserExists(socialSecurityNumber, address);
  if (application.isAdministrator()) {
    roleDao.makeAdministrator(socialSecurityNumber);
  }
  return new ModelAndView("users/create", "message", "User created");
}

In the real world, we shouldn't stop here - the method could easily be split up further. However, for this blog post, we'll stop with the refactoring now.

Anyway, with that code, we'll need to mock these method calls:

applicationDao.get(...), returning an Application
applicationDao.grant(...)
userDao.ensureUserExists(...)
roleDao.makeAdministrator(...)

Small change? Well, the code no longer needs to know how to change the state of the application, how to make sure a user exists in the database, or how to promote a user to administrator. As a result, the cyclomatic complexity of the new code is much lower, so just two tests will cover the positive cases. Of course, we've introduced new methods in the other services which need to be tested in their respective test suites, but each of them should be pretty straight-forward to test.

And in fact, we've done only two things in the refactoring:

  • Tell, don't ask. Instead of fishing out values and passing them back and forth, we've defined new methods that just "do the job", without us having to figure out how to meddle with a lot of object in the same method.
  • Flesh out responsability. In our method, we don't want to know that in order to promote a user to administrator, we need to give the user a certain role. That's what RoleDao knows, and there's no need to spread that knowledge to all our classes.

These simple tools are great when you're trying to decouple your components, and incidentally, you end up with more testable code.

And even better, if you remember these tools when you're test-driving your code, you'll start off writing nice, decoupled code.

Don't be afraid of writing a lot of small classes with their own, very limited responsability. Each of them will be easy to test, and your code base will be more readable.

Thursday, December 16, 2010

HD HomeRun, MythTV, DVB-C

HD HomeRun
Efter at have kørt med en Anysee-USB-tuner i et års tid i mit MythTV-setup (se http://olefriis.blogspot.com/2010/06/mythtv-on-eeebox.html), var jeg ved at være rigtig træt af den. Pludselig, uden varsel, begyndte den at have problemer med at tune ind på bestemte kanaler - det kunne tage 30-40 minutter for den at tune ind. Og når den endelig var tunet ind, kunne det tage tilsvarende 30-40 minutter at skifte væk fra disse problematiske kanaler. Problemet forsvandt pludselig igen, men dukkede senere op igen - uden at jeg havde opdateret software. Se fx http://www.mythtv.dk/viewtopic.php?id=109 og http://www.mythtv.dk/viewtopic.php?id=169.

Et par kollegaer havde købt en HD HomeRun og var meget glade for den, så det skulle jeg også prøve. Jeg købte den i MM-Vision for 1.499,-. Det er vist ikke meget dyrere end man kan få den til i udlandet.

Desuden er der noget frustrerende over at Anysee hverken officielt eller i praksis supporterer deres tunere på Linux. Det er en finne der på eget initiativ har sørget for understøttelse af tuneren i kernen, hvilket er et super-initiativ, men der er meget langt derfra og så til officiel understøttelse af hardwaren fra det firma der tjener penge på salget.

Det er rigtig nemt at sætte en HD HomeRun-enhed op, når lige man kender til et par detaljer. Så jeg tænkte at jeg ville skrive disse detaljer ned, så andre måske kan få glæde af dem.

"The basics"
Du må selv sørge for at sætte HD HomeRun-enheden på dit netværk og sørge for to antennestik til den :-) Så skal du også sørge for at opgradere firmwaren på enheden. Jeg tror nok det sker af sig selv hvis man bruger Ubuntu og starter hdhomerun_config_gui, men jeg er ikke sikker - selv sad jeg på min Mac. Jeg hentede nyeste tools (http://www.silicondust.com/support/hdhomerun/downloads/), og ifm. installationen på min Mac blev firmwaren opdateret.

Du skal nok sørge for at få både HD HomeRun-enheden og din MythTV-boks på kablet net, da det er ret vigtigt at forbindelsen de to imellem ikke er for "shaky".

I hdhomerun_config_gui skal du sætte begge tunerne til "eu-cable".

Hvis du har en Anysee-tuner
Man kan godt få Anysee-tuneren og HD HomeRun-boksen til at leve i fred og fordragelighed, men jeg valgte bare at fjerne min Anysee-tuner i opsætningen af MythTV og hive antennestikket ud af den. Vil du have de to til at fungere side om side, skal du efter sigende lave en ny "input source" i backend-opsætningen. Ellers prøver MythTV at skifte til HD HomeRun-enhedens kanaler på Anysee-tuneren, hvilket Anysee-tuneren bare bliver forvirret over.

Jeg ryddede alle eksisterende kanaler i MythTV-opsætningen, så tavlen var visket ren.

Nu fungerer Anysee-tuneren blot som IR-receiver.

DVB-C
Som udgangspunkt kører HD HomeRun DVB-T. Det er måske også fint for dig, men hvis du har kabel-tv, er det ikke godt nok. Først skal du kende id'et på din HD HomeRun-enhed. Det gøres ved at køre følgende på kommandolinjen:
hdhomerun_config discover

Det gav i mit tilfælde 12106FA4, som jeg derfor bruger nedenfor.

Har du Stofa, ligger alle kanaler (mig bekendt) på symbolfrekvens 6900000, modulation QAM 64 og 256. Det skal HD HomeRun have at vide:
hdhomerun_config 12106FA4 set /sys/dvbc_modulation "a8qam64-6900 a8qam256-6900"

For Yousee skal du gøre følgende i stedet:
hdhomerun_config 12106FA4 set /sys/dvbc_modulation "a8qam64-6875"

Se den officielle support-side for flere informationer: http://www.silicondust.com/support/hdhomerun/instructions/dvb-c/

Opsætning i MythTV
Denne del er rigtig nem, idet MythTV allerede kender til HD HomeRun. Så bare vælg denne type i backend-opsætningen under opsætning af enheder. Du skal oprette en enhed til hver af de to tunere der sidder i boksen.

Desuden skal du (hvis du ønsker programinformation på kanalerne) oprette en "input source" og sætte denne på de to enheder. Da de to enheder nu har samme "input source", kan MythTV vælge frit blandt de to tunere til en given optagelse, og du kan optage flere programmer fra flere modulationer samtidig.

Kanalsøgning
Naturligvis kan man ikke "bare" gå ind og lave en fuld kanalsøgning i MythTV - man skal eksplicit angive hvilken frekvens og QAM man ønsker at søge på. Men først skal du selvfølgelig vide hvilke frekvenser og modulationer dine kanaler ligger på:
hdhomerun_config 12106FA4 scan 0 kanaler.txt

Lav så en kanalsøgning i MythTV-backend-opsætningsprogrammet på hver af de frekvenser der var "hit" på i kanaler.txt - du skal eksplicit angive både frekvensen og QAM'en. Med Stofa fandt jeg dog alle kanaler (i den lille pakke jeg har) ved at søge på frekvens 434000000 Hz, QAM256.

Efterfølgende skal man selvfølgelig ind at rette xmltv-id'erne til, så din xmltv-import hælder programinformation på de rigtige kanaler. Efter en ny "mythfilldatabase" skulle alt nu se fint ud!

Saturday, June 12, 2010

MythTV on an EeeBox

As many other geeks, I've spent considerable time in my life setting up MythTV and trying really hard to set it up for the perfect media center experience. I'm still struggling with some points, but I've got it working pretty decently, so I thought I'd post the result of my process, along with the still quirky points. Then hopefully somebody can use it for setting up a decent media center, and if I'm really, really lucky, some of you über geeks may have a few tips to share in the comments, so I can iron out the last weird issues.

Please note: I live in Denmark, so certain points in this post may not be relevant to you (especially the XMLTV guide). For that, you'll have to consult other blogs or - shiver! - some actual documentation.

What I bought
To get up and running, I bought the following hardware:
  • EB1012 (an Asus EeeBox with the ION graphics chipset, no optical drives)
  • Anysee EC30 Combo Plus DVB-C/T (a USB tuner supporting both DVB-C and DVB-T, and it comes with a remote control)
Apart from that, I bought an antenna splitter and the shortest, cheapest HDMI cable I could find.

Characteristics of the system
There are thousands of possible setups for running MythTV, and many people have "enterprisey" setups with a monster back-end and several front-ends. So why did I choose this simple setup?
  • Decent power consumption. With everything plugged in and running, expect an average power consumption of about 22W (excluding your TV and other stuff, of course).
  • Discrete. The EeeBox is very quiet, and it hides nicely behind my TV. The Anysee tuner "peeks out" from below, exposing the IR receiver.
  • Enough CPU and GPU power to easily play back all the contents I need to play, including HD content.
  • HDMI, separate digital optical audio (note: the EeeBox comes with a converter to plug into the jack plug at the back of the computer, so you don't need to buy one), enough USB ports, card reader, 802.11n wireless (and gigabit ethernet if you want), eSATA, ...
  • MythTV has a lot of nice features, and Ubuntu lets you fiddle with everything.
The EeeBox "only" has a 250GB harddisk, which may be too small for you. For me, it works, since I like to delete old, watched recordings anyway (and delete the ones I accept that I won't see anyway).

MythTV isn't for the faint-hearted, which this guide should make clear. I expect you to have basic Linux skills, otherwise this guide is probably useless, and in that case you should go for something simpler like EyeTV on a Mac or Windows Media Center on a Windows box.

Installation
Get Mythbuntu 10.04 from mythbuntu.org. Put it on a USB key, and make this USB key bootable.

Well, that's just not as easy as I thought it would be. There are lots of blogs out there telling you how to do it, but the only way I got it working, was with the help of a little Windows program called "Linux Live USB Creator" (http://www.linuxliveusb.com/). It does not recognize Mythbuntu directly, but it supposes it's an Ubuntu variant, which is correct.

Anyway, when you have your bootable USB key ready, put it in one of the USB ports on the EeeBox, and power on the EeeBox while holding down the F8 key on the keyboard.

How you choose to partition your hard disk is entirely up to you. I chose 20GB for the root partition, 5GB for swap, and the rest mounted as "/data". Other values will probably fit your needs better.

When prompted, choose having a combined front-end and back-end.

Update!
When your new MythTV box has booted for the first time, open "Mythbuntu Control Centre", go to the "Repositories" page, and select updates from "0.23 fixes". After that, run Update Manager, install all updates, and reboot.

Without this step, several components of MythTV didn't work properly for me.

Storage paths
Before you start using MythTV for real, remember to adjust the paths in MythBackend that define where recordings, live-tv, etc. go. It's under "Storage Directories". If, like me, you've mounted most of the EeeBox harddisk under "/data", you probably want values such as "/data/mythtv/recordings", "/data/mythtv/livetv", etc.

You should also create these directories in the file system, and remember to give proper rights to the mythtv user. I've just set all these directories to the mythtv user, the mythtv group, and given all rights possible. Perhaps that's a bit too much, but I don't want to spend time fiddling with this.

Drivers
Open "Mythbuntu Control Centre" again, go to the "Graphics Drivers" page, click "Launch Restricted Drivers Manager", and select both "Firmware for DVB cards" and "NVIDIA accelerated graphics driver". You'd think you didn't need the firmware for DVB cards, but without it, you cannot make the Anysee remote control work later.

Tuner in DVB-C mode
(I'm not sure if this first step is necessary - I'll check it out next time I re-install from scratch). Run this from a terminal:

modprobe dvb_usb_anysee delsys=0

Unfortunately, there is both a DVB-C driver and a DVB-T driver recognizing the tuner, so you'll still have trouble using the tuner after this. To fix this, you have to blacklist one of the drivers. In this case, we want to blacklist the DVB-T tuner. Add the following lines to /etc/modprobe.d/blacklist.conf:

# Force Anysee tuner into DVB-C mode
blacklist zl10353

If you want to run in DVB-T mode instead, you have to blacklist "tda10023" instead.

Setting up the tuner in MythTV
The card type must be set to "DVB DTV capture card (v3.x)", and the DVB Device Number should be set to "/dev/dvb/adapter0/frontend0". The next line should then tell you that "Frontend ID" is "Philips TDA10023 DVB-C Subtube DVB-C".

I've set my "Signal Timeout" to 5000ms and "Tuning Timeout" to 20000ms.

Channel search and setup
It's somewhat of a circus setting up channels, since subsequent, identical searches can yield different results. My only advice is to try a few times and then hope you can stitch together your channel setup. Try setting "Signal Timeout" and "Tuning Timeout" high (as I've done - see above).

I use a Danish provider called Stofa, and I can find all of their channels on frequency 346000000, modulation QAM64, symbol rate 6900000. This is probably different for your setup.

Sometimes MythTV tells me it has found a number of "off-air channels". I've no idea what that means, but typically they are channels already found as regular channels anyway.

XMLTV
In Denmark, the main national TV provider, DR, has a service that lets you grab the program information of the most used channels. You can use a grabber like "http://niels.dybdahl.dk/xmltvdk/index.php/DR_2009_grabber".

The XMLTV Perl package is installed by default in Mythbuntu, but you need to install "libparse-recdescent-perl". Then you can put the grabber script in your home directory and create the following script:


#!/bin/bash
LOGFILE=/home/me/mythfilldatabase.log
source /home/me/.profile
mv /var/log/mythtv/tvgrabbed.xmltv.1 /var/log/mythtv/tvgrabbed.xmltv.2
mv /var/log/mythtv/tvgrabbed.xmltv /var/log/mythtv/tvgrabbed.xmltv.1
cd /home/me/
./tv_grab_dk_dr_2009.mgr --days 10 --output /var/log/mythtv/tvgrabbed.xmltv > ${LOGFILE} 2>&1
/usr/bin/mythfilldatabase --file 1 /var/log/mythtv/tvgrabbed.xmltv >> ${LOGFILE} 2>&1

Then set up a Cron job: Run "crontab -e", define a line like:

00 06 * * * /home/me/mythfilldatabase.sh

Additionally, you ned to start "mythtv-setup" and define a new "Video Source" with the name "dr-xmltv" and "No grabber" as "Listings grabber". Under "Input Connections", this video source must be connected to the tuner.

Finally, run your new script. Then in MythWeb you can insert the right XMLTV IDs on the corresponding channels. Run the script again, and you get a proper channel listing in MythWeb.

Sound via HDMI or TOSLINK
If you want sound through the HDMI cable, create the file /etc/asound.conf with the following contents:


pcm.!default {
  type hw
  card 0
  device 3
}


If instead you want sound through the optical digital out, specify device 1.

Start "alsamixer" from a terminal and unmute "S/PDIF 1" by clicking M on it.

Picture quality
Turn on VDPAU in the "Setup -> TV Settings -> Playback", otherwise the EeeBox doesn't stand a chance when playing HD content.

Enable "OpenGL Vertical Sync" under "Setup -> TV Settings -> Playback". Otherwise you get "tearing" in the picture (i.e., the content is updated in the middle of the picture, so you have one part of the picture showing the previous frame, and the rest is playing the next frame).

Still, playback isn't always perfect, since the refresh rate on your TV doesn't necessarily match the refresh rate of the played material. To fix this, you need to do two things: First, go to "Setup -> Appearance", and on the "Video Mode Settings" page, check "Separate video modes for GUI and TV playback". Just choose "Any" as the rate for the GUI, and don't change the values in the "Overrides for specific video sizes" section.

Secondly, open nvidia-settings and uncheck the "Force Full GPU Scaling" on one of the pages. Voila! Playback is now perfect! (At least on my setup :-) )

The remote control
First and foremost you need to open "Mythbuntu Control Centre", go to the "Infrared" page, and check "Enable a Remote Control". Choose "Linux input layer (/dev/input/eventX)". If you cannot find this, it's because you didn't install the proprietary drivers - see above!

Run "cat /proc/bus/input/devices" from a terminal. Near the bottom of the output you'll see something like this:


I: Bus=0003 Vendor=1c73 Product=861f Version=0100
N: Name="IR-receiver inside an USB DVB receiver"
P: Phys=usb-0000:00:04.1-6/ir0
S: Sysfs=/devices/pci0000:00/0000:00:04.1/usb1/1-6/input/input6
U: Uniq=
H: Handlers=kbd event6
B: EV=3
B: KEY=148fc010 2353041 0 0 0 800 108000 4080 304801 9e0000 0 0 ffc

From the "H:" line, you can see that the remote control exists on "/dev/input/event6" - it is probably different on your system, since it depends on which USB port your tuner is in, and which other peripherals you have attached to your EeeBox.

Edit "/etc/lirc/hardware.conf", change the line

REMOTE_DEVICE="/dev/lirc0"

to

REMOTE_DEVICE="/dev/input/event6"

Then you can restart lirc with "sudo service lirc restart".

There is one real downside to doing this: Each time you reboot your EeeBox, you must have the same peripherals attached, otherwise your remote control will appear as another path. I haven't fixed this, but it should be fixable by doing this: Create "/etc/udev/rules.d/60-anysee.rules" (owner root, group root, "-rw-r--r--", just like the other files in that directory) with contents:

ATTRS{name}=="IR-receiver inside an USB DVB receiver", SYMLINK+="input/dvb-ir"

Now the remote control (also) appears as "/dev/input/dvb-ir", so you can set REMOTE_DEVICE in the lirc hardware.conf file to that. However, lirc refuses to use it, and even worse, as long as the udev rule exists, it even refuses to use "/dev/input/event6". So, I need to attach my keyboard and mouse each time I reboot my EeeBox...

Key bindings for the remote control
The remote control's "|<<" and ">>|" buttons are not "mapped" to anything in MythFrontend. I've chosen to let them work as "rewind" and "fast forward", respectively. You do this by opening "~/.lirc/mythtv" and insert the following lines:


begin
  remote = devinput
  prog = mythtv
  button = KEY_NEXT
  config = Right
  repeat = 0
  delay = 0
end

begin
  remote = devinput
  prog = mythtv
  button = KEY_PREVIOUS
  config = Left
  repeat = 0
  delay = 0
end

Playback in MythWeb
A really nice "gimmick" in MythWeb is the Flowplayer media player, which gives a YouTube-like interface to your recordings. In MythWeb, click the "MythTV:" link (upper left corner), select "Settings -> MythWeb -> Video Playback", and select "Enable Video Playback".

Securing MythWeb
There are several guides "out there" that tell you how to fiddle with stuff in /etc/apache2, /etc/httpd, etc. None of those guides seem to work... but that doesn't matter, since you just have to open "Mythbuntu Control Centre" (start it as root), go to the "Plugins" page, and enable password protection.

Start/end of recordings
It's nice to start recording e.g. 2 minutes before the official starting time, and end recording e.g. 5 minutes after the official ending time. In MythWeb, you can alter the setting "DefaultStartOffset" and "DefaultEndOffset". Both are specified in minutes, so I've set them to 2 and 5, respectively. This implies that the default values of the fields "Start Early" and "End Late" in the recording dialog page will be set to 2 and 5 minutes.

Issues still not sorted out
My daily Cron job doesn't always run. However, it successfully runs often enough that I don't have problems with my channel listing.

When I reboot the EeeBox, I need to have my keyboard and mouse attached, otherwise the remote control won't work. It seems like udev and lirc don't like each other.

Update (25th of August 2010): My setup has been working quite badly the last month or so, since the back-end sometimes takes 30 to 40 minutes to change the channel. I've borrowed another Anysee tuner from a friend, but that didn't fix the problem, so either my cable provider is sending a poorer signal (which would be weird, since my TV has no problem), or a bug has been introduced either in MythTV or the kernel driver. At around the same time, the 0.23-fixes branch was ended, and no Mythbuntu updates switched to the 0.23.1-fixes branch. I've switched to the 0.23.1-fixes branch by executing this from the command-line:

sudo add-apt-repository ppa:mythbuntu/0.23.1
sudo apt-get update

I haven't updated MythTV yet (it takes proper planning in a busy household...), so I've yet to see if this helps.

References
Helping me get this far required a number of resources on the net. In all fairness I'll list them here: