University Hackathon Olympics

Earlier this month, Lookout sponsored the University Hackathon Olympics, a 24-hour event that brought talented college students and industry professionals together to hack on ideas. Over the course of the event, some teams hacked on hardware, others built new ways to explore data, and a handful prototyped some great ideas.

Here are some photos from the event:

Team B Miner B Miner built a prototype that mines bitcoins using donated AWS credits
Team Code Goggles Code Goggles visualized complex SQL queries
Team Dog Ear Dog Ear built a Chrome extension that tracks and shares what you've watched online with your friends
Team Mailicorn Mailicorn built a JavaScript interface for managing multiple email accounts
Universal Physics Solver This team built a universal physics solver
Sau Hill Ventures Sau Hill Ventures built a tool that helps merchants accept bitcoins

Hackathons have been part of Lookout’s culture from the very beginning. Engineers at Lookout hack regularly by building prototypes or releasing software to the open source community. Our hackernoons are a great time to explore new ideas, build demos, get feedback, and ultimately make things better.

Since we believe so strongly in hackathons, it should come as no surprise that we were eager to volunteer as mentors to help students productively hack on their ideas. We worked with Project Shepherd, a large and ambitious group of students interested in building a missing person finder. Project Shepherd is open sourced on GitHub.

We had a great time. Here are a couple of photos of the team in action:

Brainstorming ideas Brainstorming ideas
The list of ideas The list of ideas
Whiteboarding the user interface Whiteboarding the user interface
Hacking Hacking
Practicing the pitch Practicing the pitch
The Shepherds The Shepherds

Want to learn more? - Learn about what it’s like to work at Lookout - Check out the open jobs in Engineering - Check out all the open jobs at Lookout

Join us and help us keep millions safe.

- Marc Chung

posted in: · · ·

End-to-End Testing in Android with ImperiusGeorge

TL;DR; check out ImperiusGeorge and ImperiusGem on github and drive Android UI-Automator remotely in ruby!

Android is awesome. But its diverse ecosystem can be insanely tough to target. There are dozens of manufactures, hundreds of carriers, thousands of models, several OS iterations and an inordinate amount of OEM ‘improvements’ (touchwiz, sense, etc). There are many tools to apply automated testing to make development easier and more stable like Robotium, Robolectric and now UI-Automator. We’ve made a new way to run UI-Automator to achieve the holy grail of testing: end-to-end tests on production phones with production builds.

Ho, UI-Automator!

UI-Automator is Google’s new UI testing tool that’s included in 4.1+ phones. It’s powerful, well-integrated, and unlike Robotium it runs on production builds of all apps on production android phones!

Its problem is flexibility. Tests must run junit-style and don’t have filesystem access (there is a workaround) or package-manager or application-manager access. It is possible to use the Apache network libraries and talk to servers– potentially doing end-to-end testing but there are a couple drawbacks. You can’t quickly and easily install and remove apks or use many of the developer tools available via ADB. It’s also difficult to coordinate multiple devices.

We wanted a way to keep our entire test suite in Ruby, running on a coordinating computer with adb access and multiple connected devices.

funny logo

##Enter ImperiusGeorge We wrote a tool to let us write Ruby on that runs on a coordinating computer that controls any UI-Automator enabled device over USB. You write ruby on your desktop/server and it’ll drive the java runtime on Android.

It’s called ImperiusGeorge after the Imperius Curse and the adorable Curious George character. It’s remote mind control like imperio and curiously amazing like our monkey friend. Forget having testing monkeys manually verifying your project, run an unforgivable curse on Curious George instead!

Instead of writing this in Java, compiling, pushing to your phone, then running via ADB:

Instead write this in ruby alongside your server interaction code:

How it works

We needed a way to drive the Java runtime via text HTTP Get requests. Most of the work is done in Here’s where we make a single method run introspectively run any available method on any class or object with any arguments. run takes three String parameters, ‘on’, ‘method’, and ‘arguments’.

  1. on first tries to find a class via the findClass() method (It’ll work if it’s a fully-qualified class name like ‘’). If that succeeds it’ll run The method statically.
    • If the method is ‘new’ it’ll run the constructor it can find with the given arguments.
    • If it can’t find the class it’ll try looking up the instance (see step 3)
  2. It then loops through each method available, finding ones with the same name and matching number of arguments.
    • adaptArgs() then tries to adapt, convert, or cast the provided arguments to the method’s contract
    • If it throws an IllegalArgumentException it tries to match more methods (to support overloading).
  3. Finally it calls the method and calls adaptReturn() on the result
    • If the returned value is a java.lang.* object it’ll json-serialize it, returning it back to ruby.
    • A planned improvement is to also return serializable arrays and maps.
    • If the object is not directly returnable it stores it in a HashMap<String,Object> and returns a unique hash key. That’s how the third option for the ‘on’ parameter works.

A note about bundled code: UI-Automator is actually a system tool, invoked via adb shell. It loads compiled jars’ classes and introspectively runs the parameter passed class (it’s junit extended). Because of this architecture you can’t include separate libs (oh, we tried). We wanted to use gson but chose simple-json for simplicity. We also used NanoHTTPD for the web server functionality. These libraries simplified development a lot! We bundled the source to help simplify yours too, it’ll compile and run as-is.

We, for one, welcome our new overlords

At Lookout we’ve put this functionality into our checkin verification process on jenkins. If you write something on the client it’ll run an unsigned production build on real devices first before merging.

- Tim O’Brien

posted in: · · · · ·

The Frankenjar: The Easiest Ruby App Deployment Ever

I’ve had a love/hate relationship with Java and in turn the JVM. I love to hate them both. Begrudgingly, I can acknowledge the performance and maturity of the JVM, especially when compared with the simplistic Ruby VM (MRI). Historically, Lookout has been a very Ruby-oriented company, I would estimate that 80%+ of our server-side code is Ruby-based (including Chef). It’s tremendously easy to build Ruby projects at Lookout, but the deployment process and production performance leave something to be desired.

When my team started a new project, we looked at the status quo of the Ruby VM and decided that JRuby was the best, most logical environment to use in production. But for local development, the quick boot-up time and adequate speed of MRI (1.9.3) made it the better choice. From the very beginning our tests were running in Jenkins under MRI 1.9.3 and JRuby 1.7.

As the project matured, we needed to address the deployment question. The application consisted of Ruby code, and a modern front-end application developed using Chaplin (an application framework built atop Backbone.js). Our needs and wants were:

  • We need to deploy continuously
  • We need easy rollbacks
  • We need to pre-compile the front-end application from CoffeeScript and Stylus to minified JavaScript and CSS.
  • We want to reduce the complexity of the Operations work involved to run the application
  • We want to run the application with JRuby
  • We want as much control over the environment in production as possible
  • We need to be able to run an interactive console for debugging

After an afternoon of tinkering with Warbler, a gem which makes .jar/.war files from Ruby applications with JRuby, we were able to create the “frankenjar.” The frankenjar is a self-contained, self-running version of the entire application.

Creating the jar

The first of the two kinds of self-running files that Warbler will create is a .war file that embeds a Java Servlet container. Using Jetty or Winstone, combined with the jruby-rack gem to glue a standard Rack application to these servlet containers.

The .war file is a perfectly acceptable way to run the web application, but we wanted more than just a web server from the executable.

This led to creating an executable .jar file with the Rack application and other tools packaged up together, such that we would end up with a single, traceable archive at the end of the process.

The first step to building a frankenjar is defining a config/warble.rb file. The file contains two important pieces of information: the directories/files to bundle up, and the name of the resulting archive.

Warbler does some helpful auto-detection for Rack applications which gets in our way in this case. Since we do have a Rack application, but we don’t want to create a .war file, we need to “hide” from Warbler during the packaging process. To help with this we created the following script to properly pre-compile our front-end assets, and create the .jar file.

Executable jars created by Warbler will automatically invoke the first file listed in the source tree’s bin/ directory when the .jar file is run. We can use this to our advantage and provide a simple task invocation mechanism in that file.

After an hour or two of cobbling something together “manually”, I decided to just use Rake as the task invocation mechanism. By using Rake we are able to re-use the same exact task definitions that we’re using locally for performing tasks such as database migrations, starting up test consoles, or even running the server itself.

Coaxing Rake to properly load the Rakefile embedded within the .jar file was a bit of a challenge, and does require monkey-patching Rake:

With all that put in place, invoking ./scripts/ will create a nice, entirely self-contained, runnable .jar file which can be shuffled around as needed.

Running the jar

The system requirements for running the application are dead-simple: the Java Runtime Environment. The Chef code required to set up machines for the application was incredibly simple: install the JRE, make sure a Java process runs forever, rotate logs as needed.

Within the .jar file, we’re pretty much boot-strapping with Rake, the command line interface should look familiar:

-> % java -jar franken.jar -T
(in /home/tyler/source/lookout/git/frankenjar)
rake console:pry     # Run a simple pry-based console
rake db:migrate      # Migrate the database
rake db:rebuild      # Rebuild the entire database
rake db:schema:dump  # Dump the new schema to the terminal
rake server:puma     # Run the application with Puma
rake server:webrick  # Run the application with WEBrick (development only)
-> %

There are many other Rake tasks in the project, but most of them are related to local development and testing. it should be noted that in the config/warble.rb above only includes a select few .rake files to be bundled inside the archive, no sense running unit tests in production.

Because we’re using the same Rake tasks, we’re getting a bit of extra testing and usage of those tasks and codepaths that will be used locally and in production.

Deploying the jar

As you might imagine, shipping a single .jar file to production is rather simple, as it should be. Where the additional complexity comes in is during the updating of the running app instances. While traditional .war containers will support hot restart, we have to manage this ourselves to support zero-downtime deployments.

Unfortunately the JVM warm-up period, and loading the entire runtime can take between 5-15s depending on load, so we walk through app servers, restart them, and block until they’re back online. To accomplish this we use a little bit of Capistrano:

desc 'Restart server'
task :restart_apps, :roles => :app do
  find_servers_for_task(current_task).each do |server|
    puts "Running a server restart on #{}"
    run "#{sudo} /srv/frankenjar/", :hosts => []

With a little bit of shell scripting, to block the deployment until the frankenjar is serving requests again, we can rely on the upstream load balancer (nginx) to handle the fail-over to other app instances during the restart.

It’s Alive!

With the frankenjar being created as the last step of our internal build and testing pipeline, we take .jar files from Jenkins and deploy them just about anywhere with minimal overhead. The .jar is the ultimate reproducible artifact, developers, QE, and operations can all access and use the exact same code when reproducing any issues that may arise.

The approach allows the developer team to upgrade JRuby, any dependent gems, switch out web servers, and reuse all the powerful profiling tooling available in the Java community when doing it.

While the frankenjar is slightly off the beaten path in the JRuby community, it’s definitely worth the extra little bit of work to get there.

- R. Tyler Croy

posted in: · · · · · ·