Sign in Subscribe
By tanin in tech —

Integrate JobRunr into Play Framework

This blog shows you how you can integrate JobRunr with Play Framework.

JobRunr is one of the popular background processing for a JVM application, but it is catered toward Java and Spring Boot. Since there's no popular Scala background processing framework, and I don't want to just roll out my own, I've decided to use JobRunr.

This is what Scala intends: to be able to utilise a Java library and framework. This is one of the main reasons why Scala is on JVM and interoperable with Java.

JobRunr's quick architecture overview

JobRunr has 4 main components: the background job runner (for some reason, it is called "backgroundJobServer"), the scheduler (who adds a new job request), and the admin dashboard.

  1. Job requests and their handlers are the jobs that you want to queue and to be done by JobRunr.
  2. The scheduler is an instance in your application where you can schedule a job request. When doing so, a job request would be added to JobRunr's database table.
  3. The admin dashboard exposes the web interface for debugging and for other controls (e.g. triggering a recurring job).
  4. The background job runner (or "backgroundJobServer") runs on its own process. It would poll for new jobs from the database and run the jobs when appropriate.

JobRunr uses a database, and you would have to set up the database connection properly. However, it completely manages its own database tables out of the box. You wouldn't need to do anything.

Set up a job request and its handler

Setting up a job request is straightforward but there are 2 things to look out for:

  1. A job request should be serializable because it will be serialized and deserialized in and out of a database.
  2. A job request handler needs to play well with Play's injection framework because it might need to inject other important things like WSClient or Slick-related code for reading and writing database.

Here's a simple request and its handler: 

case class ProcessUpdateRequest(updateId: String) extends JobRequest {
  def getJobRequestHandler = classOf[ProcessUpdateRequestHandler]
}

@Singleton
class ProcessUpdateRequestHandler @Inject() (
  wsClient: WSClient, 
  app: Application
)(implicit ec: ExecutionContext)
    extends JobRequestHandler[ProcessUpdateRequest] {

  def run(req: ProcessUpdateRequest): Unit = {
    // do something...
  }
}

Notice that ProcessUpdateRequestHandler is working within the Play's injection framework and is able acquire the instance of Application.

Initialisation using Play Modules

If you depend on Play Framework a lot, you will need to learn to master Play Modules. A Play module can be used to initialise instances of classes and register them with the Play's injection framework, which uses Guice underneath I think.

The main instance that we want to initialise is JobRequestScheduler, which is used for scheduling a job request to be worked on.

Let's set up a module first by making JobRunrModule and adding it to application.conf :

play.modules.enabled += "modules.JobRunrModule"

Next, we want to make an injectable JobRequestScheduler, so you will be able to use it in various places inside you Play application. JobRequestScheduler requires StorageProvider, which in turn requires a database URL.

I'd recommend separating out StorageProvider into its own injectable instance because it'll be used by the background job runner as well.

Here's the full code: 

@Singleton
class StorageProviderProvider @Inject() (
  config: Configuration,
  lifecycle: ApplicationLifecycle
)(implicit
  ec: ExecutionContext
) extends Provider[StorageProvider] {
  lazy val provider: StorageProvider = {
    val dataSource = new DatabaseUrlDataSource()
    dataSource.setUrl(config.get[String]("slick.dbs.default.db.properties.url"))
    
    val config = new HikariConfig()
    config.setDataSource(dataSource)
    
    val provider = SqlStorageProviderFactory.using(new HikariDataSource(config))

    // Ensure Play dev's hot reloading cleans it up correctly.
    lifecycle.addStopHook(() => Future { provider.close() })

    provider
  }

  def get(): StorageProvider = provider
}

@Singleton
class JobRequestSchedulerProvider @Inject() (
  storageProvider: StorageProvider,
  lifecycle: ApplicationLifecycle
)(implicit
  ec: ExecutionContext
) extends Provider[JobRequestScheduler] {
  lazy val scheduler: JobRequestScheduler = {
    val scheduler = JobRunr
      .configure()
      .useStorageProvider(storageProvider)
      .initialize()
      .getJobRequestScheduler()

    lifecycle.addStopHook(() => Future { scheduler.shutdown() })

    scheduler
  }

  def get(): JobRequestScheduler = scheduler
}

class JobRunrModule extends AbstractModule {
  override def configure(): Unit = {
    bind(classOf[StorageProvider])
      .toProvider(classOf[StorageProviderProvider])
    bind(classOf[JobRequestScheduler]).toProvider(
      classOf[JobRequestSchedulerProvider]
    )
  }
}

FYI, we are using Slick. Therefore, we reuse the database URL which is set for Slick.

Now you should be able inject JobRequestScheduler into any place you would like to use it. For example, here's how you can use it in a controller:

class TestController @Inject() (
  jobScheduler: JobRequestScheduler, 
  cc: ControllerComponents
) extends AbstractController(cc) {
  def index = Action {
    jobScheduler.enqueue(ProcessUpdateRequest("some_id"))
    Ok("It works!")
  }
}

Initialise JobRunr's admin dashboard

Using a 3rd-party framework like JobRunr is nicer than building my own framework because they also offer an admin dashboard, which would have been tedious to build myself.

You can easily spin up the JobRunr's admin dashboard (which comes with a built-in HTTP server) using the following code:

@Singleton
class JobRunrDashboardWebServerProvider @Inject() (
  storageProvider: StorageProvider,
  lifecycle: ApplicationLifecycle
)(implicit
  ec: ExecutionContext
) extends Provider[JobRunrDashboardWebServer] {
  lazy val config = JobRunr
      .configure()
      .useStorageProvider(provider)
      .useDashboard(
        JobRunrDashboardWebServerConfiguration
          .usingStandardDashboardConfiguration()
          .andBasicAuthentication("some-username", "some-password")
          .andPort(8000)
      )

  lazy val dashboard = {
    val field = config.getClass.getDeclaredField("dashboardWebServer")
    field.setAccessible(true)
    
    val _dashboard = field.get(config).asInstanceOf[JobRunrDashboardWebServer]
    lifecycle.addStopHook(() => Future { _dashboard.stop() })
    _dashboard
  }

  def get(): JobRunrDashboardWebServer = dashboard
}

class JobRunrModule extends AbstractModule {
  override def configure(): Unit = {
    ... other code ...
    
    bind(classOf[JobRunrDashboardWebServer])
      .toProvider(classOf[JobRunrDashboardWebServerProvider])
      .asEagerSingleton()
  }
}

FYI, the dashboard instance isn't a public field, but we need to stop the dashboard server when Play is shutting down. Therefore, we use Java's reflection to retrieve the instance of the dashboard server.

asEagerSingleton() is an important line because this would immediately instantiate JobRunrDashboardWebServer as opposed to using lazy loading. Since nobody requires the dashboard server, lazy loading would never be triggered.

Set up the background job runner (aka BackgroundJobServer)

This background job runner should run in a separate process. Therefore, we must make a new main class. We'll name it: JobRunrMain .

The main class should instantiate a Play instance like below: 

object JobRunrMain {
  def main(args: Array[String]): Unit = {
    val app = GuiceApplicationBuilder(
      environment = Environment.simple(mode = Mode.Prod)
    ).build()

    Play.start(app)

    JobRunr
      .configure()
      .useStorageProvider(app.injector.instanceOf[StorageProvider])
      .useJobActivator(new JobActivator {
        def activateJob[T](tpe: Class[T]): T = {
          app.injector.instanceOf[T](tpe)
        }
      })
      .useBackgroundJobServer(1)
      .useJmxExtensions()
      .initialize()

    Thread.currentThread().join()
  }
}

You can run it in the dev mode with: sbt 'runMain JobRunrMain' . If you use sbt-native-packager , you can run it with: ./bin/<your-app> -main JobRunrMain.

Please notice that JobActivator is responsible for retrieving a job request handler through Play's injection framework. In our case, T would be ProcessUpdateRequestHandler. This is how everything is tied together.

Two JobRunr's gotchas

  1. If you set up a recurring job, do not use Duration. We've found that the clock is reset on every restart; a deploy requires a restart. If you want a job to run every hour, you should use the cron expression instead.
  2. Avoid using object as an enum i.e. sealed abstract class. You should use case object instead. This is because object doesn't retain its identity after being serialized and deserialized. But case object does!

Parting thoughts

In my previous life, I tried to build my own background job framework for my previous startup, but I never had time to build all the bells and whistles (e.g. admin dashboard) and battle-test it. That was expected because building a background job framework was never the main job. I'm not gonna make that mistake again. So far I'm happy with JobRunr.

Wow, this is a long blog post. But I hope this helps you learn how to integrate with JobRunr and how to use Play Modules to initialise the things you want to use.

Subscribe to tanin

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe