Iris Decentralized Messaging

Peer-to-peer messaging for backend decentralization

2 February 2014

Péter Szilágyi

European Institute for Innovation and Technology

Eötvös Loránd University, Budapest, Hungary

Babeş-Bolyai University, Cluj-Napoca, Romania

Note, these are the offline slides of the presentation. For executable codes, please check playground availability at http://iris.karalabe.com/talks.

Distributed services are complex...

Iris in a nutshell

Decentralized messaging framework for backend services

What makes it different? It's more natural!

Messaging the Iris 'way'

We're at FOSDEM... let's see it in action!

Challenge #1 – Group chat

Implement an IRC style group chat system:

Solution #1 – Broadcast

// Print all received broadcast messages
func (e *echo) HandleBroadcast(msg []byte) {
    fmt.Println(string(msg))
}

func main() {
    // Connect to the Iris network
    conn, err := iris.Connect(55555, "go-devroom", new(echo))
    if err != nil {
        panic(err)
    }
    defer conn.Close()

    // Send a dummy message every second
    for i := 1; i <= 100; i++ {
        conn.Broadcast("go-devroom", []byte(fmt.Sprint("Al Gopher: message #", i)))
        time.Sleep(time.Second)
    }
}

Rename Al Gopher in a second window 😉

Challenge #2 – Web requests

Implement a system for handling web requests:

Solution #2 – Browser

func main() {
    // Connect to the Iris network
    conn, err := iris.Connect(55555, "browser", nil)
    if err != nil {
        panic(err)
    }
    defer conn.Close()

    // Issue a dummy request every second
    for i := 1; i <= 100; i++ {
        rep, err := conn.Request("server", []byte(fmt.Sprint("Request #", i)), time.Second)
        if err != nil {
            fmt.Println("Request failed:", err)
        } else {
            fmt.Println("Web reply: ", string(rep))
        }
        time.Sleep(time.Second)
    }
}

Start some webservers and check back 😉

Solution #2 – Web server

// Format each request a bit and return as the reply
func (s *server) HandleRequest(req []byte) []byte {
    return []byte(fmt.Sprint("www-", s.id, ": ", string(req)))
}

func main() {
    // Connect to the Iris network
    conn, err := iris.Connect(55555, "server", &server{id: rand.Intn(100)})
    if err != nil {
        panic(err)
    }
    defer conn.Close()

    // Serve a while, then quit
    fmt.Println("Waiting for requests...")
    time.Sleep(100 * time.Second)
}

Start another webserver, I dare you 😉

Challenge #3 – Aperture Science Enrichment Center 😈

Implement the comlink for Aperture Laboratories¹:

¹ http://en.wikipedia.org/wiki/Portal_(video_game)

Solution #3 – GLaDOS

func main() {
    // Connect the network
    conn, err := iris.Connect(55555, "GLaDOS", nil)
    if err != nil {
        panic(err)
    }
    defer conn.Close()

    // Publish the nice wishes
    fmt.Println("GLaDOS is online...")
    for {
        // Pick a random wish
        idx := rand.Intn(len(wishes))

        conn.Publish("official", []byte("GLaDOS: "+wishes[idx]))
        time.Sleep(5 * time.Second)
    }
}

Boot GLaDOS and let the experiment begin 😉

Solution #3 – Chell

func (e *echo) HandleEvent(msg []byte) {
    fmt.Printf("%s\n\n", string(msg))
}

func main() {
    // Connect to the network
    conn, err := iris.Connect(55555, "Chell", nil)
    if err != nil {
        panic(err)
    }
    defer conn.Close()

    // Subscribe to some topics
    fmt.Println("Tuning in to Aperture channels...")
    for _, topic := range []string{"official", "portal"} {
        conn.Subscribe(topic, &echo{topic})
    }
    time.Sleep(100 * time.Second)
}

Maybe there is an "unofficial" channel? Find it in the online presentation! 😉

Challenge #4 – Transactions

Implement a distributed transaction system:

Solution #4 – Client

    // Open an outbound tunnel to a database
    tun, err := conn.Tunnel("database", time.Second)
    if err != nil {
        fmt.Println("Database connection failed:", err)
        return
    }
    // Send a multipart transaction
    for i := 0; i < 10; i++ {
        tun.Send([]byte(fmt.Sprint("Part #", i)), time.Second)
    }
    // Retrieve the database replies
    for i := 0; i < 10; i++ {
        msg, _ := tun.Recv(time.Second)
        fmt.Println(string(msg))
    }
    tun.Close()

Start some database processes 😉

Solution #4 – Database

func (d *db) HandleTunnel(tun iris.Tunnel) {
    // Make sure tunnel is cleaned up
    defer tun.Close()
    tx := d.tx
    d.tx++

    // Echo back the arriving data until the tunnel is closed
    for {
        msg, err := tun.Recv(2 * time.Second)
        if err != nil {
            break
        }
        reply := fmt.Sprintf("db-%d (tx-%d): %s", d.id, tx, string(msg))
        tun.Send([]byte(reply), time.Second)
    }
}

Start another database?

So how does this all work?

Sneak behind the scenes

Iris nodes do the heavy lifting (one/host):

Thin clients bathe in the glory:

Intermezzo 😈

Ray tracing on 384 cores! (1024x1024, 256 spp)

Single 8 core virtual machine
48 x 8 core virtual machines
² http://embree.github.io/
³ model courtesy of Martin Lubich, http://www.loramel.net

Step by step: Grand Gopharium 😋

Imagine yourself...

...visiting Gopher Town, completely clueless on what and where to do?!

Grand Gopharium:

Let's organize 'em all! How hard can it be? 😇

Basic library employees

What about calling in sick? Or a sudden surge in visitors?

Increase staff!

How can we expand from Gopher Town to Gopher Land?

Introduce departments, get some management!

Won't the collection become stale without fresh books?

Hire a supplier!

Should we be receptive to visitor suggestions?

Add an exploration chain!

Grand Gopharium 😈

RegionRank (aka Grand Gopharium) live – Waterside restaurants

Thank you

Péter Szilágyi

European Institute for Innovation and Technology

Eötvös Loránd University, Budapest, Hungary

Babeş-Bolyai University, Cluj-Napoca, Romania