Browser-Scheduler is a scheduler of browser jobs which incorporates auto-browser to support customized browser automation. Multiple jobs could be scheduled to form a complete process to automate upon a specific web site.
Add below to maven's pom.xml
:
<dependency>
<groupId>org.kquiet</groupId>
<artifactId>browser-scheduler</artifactId>
<version>X.X.X</version>
</dependency>
In Browser-Scheduler, a job is a class which inherits the designated abstract class
JobBase
and implements its own processing logic of job.
A sample job is as follows:
import org.kquiet.browserscheduler.JobBase;
public class RestartBrowser extends JobBase {
public RestartBrowser(JobConfig config) {
super(config);
}
@Override
public void run() {
//The processing logic of job goes here.
restartInternalBrowser();
}
}
It's pretty easy because the only thing needed to do is to implement the
processing logic inside method run()
. This job will be executed in a thread
managed by an internal thread pool executor.
In Browser-Scheduler, all configuration should be placed in the browser-scheduler
section of a .yml file named application.yml
. A schedule could be configured
like this:
browser-scheduler:
jobs:
- name: TestJob1
start: "2019-11-01T00:00:00"
end: "2019-12-01T00:00:00"
dailyStart: "03:00:00"
dailyEnd: "15:00:00"
interval: 10
scheduleAfterExec: true
This configuration means job TestJob1
will be executed repeatedly during
03:00:00 ~ 15:00:00
everyday with a fix delay
equal to 10
seconds in
the period from 2019-11-01T00:00:00
through 2019-12-01T00:00:00
.
A complete configuration sample:
browser-scheduler:
instanceName: BrowserSchedulerDefault
gui:
enable: false
clearLogInterval: 86400
browser:
type: firefox
maxTask: 1
pageLoadStrategy: none
headless: false
jobParallelism: 1
springComponentScanBasePackages: "org.kquiet.browserjob.impl"
jobs:
- name: RestartBrowser
enable: true
impl: org.kquiet.browserscheduler.impl.common.RestartBrowser
start: "2019-11-01T00:00:00"
end: "2019-12-01T00:00:00"
dailyStart: "03:00:00"
dailyEnd: "03:00:01"
interval: 43200
scheduleAfterExec: true
parameters:
param1: "param1_value"
Name | Default | Description |
---|---|---|
instanceName |
The instance name of browser-scheduler | |
gui.enable |
false |
Set true to enable optional monitoring GUI |
gui.clearLogInterval |
86400 |
Clear log on GUI with specified rate(in seconds) |
browser.type |
Available values are chrome and firefox . Leave it to blank if internal browser is not required. |
|
browser.maxTask |
1 |
maximum number of tasks that can be run in the internal browser concurrently |
browser.pageLoadStrategy |
none |
It controls the behavior of waiting for page loads in the internal browser |
browser.headless |
true |
true : internal browser will display its GUI;false : internal browser won't display its GUI |
jobParallelism |
1 |
It controls how many jobs could be executed concurrently at most. |
springComponentScanBasePackages |
Extra base packages for spring component scanning . | |
jobs[*].name |
The name of job | |
jobs[*].enable |
Indicate whether the job is enabled to be scheduled | |
jobs[*].impl |
Implementation class name of job | |
jobs[*].start |
The start of the absolute period in which the execution of job is allowed | |
jobs[*].end |
The end of the absolute period in which the execution of job is allowed | |
jobs[*].dailyStart |
The start of the daily period in which the execution of job is allowed | |
jobs[*].dailyEnd |
The end of the daily period in which the execution of job is allowed | |
jobs[*].interval |
The interval(in seconds) between two consecutive execution of job | |
jobs[*].scheduleAfterExec |
true : means the interval is the delay between the termination of one execution and the commencement of the next ;false : means the interval is the rate between the commencement of one execution and the commencement of the next. |
|
jobs[*].parameters |
The parameter map of job |
Note: All time-related parameters of a job are presented in local time zone.
- Get the image from docker hub:
docker pull kquiet/browser-scheduler:latest
- Prepare all the jar files of your libraries and dependencies along with a configured
application.yml
, then:- Map the library path to
/opt/kquiet/browser-scheduler/ext
when you run it, e.g.,docker run -d -v /path/to/library:/opt/kquiet/browser-scheduler/ext kquiet/browser-scheduler:latest
- Or use
docker build
to put them into/opt/kquiet/browser-scheduler/ext
to create your own image to run it without volume mapping
- Map the library path to
- How to configure a job with multiple combination of (dailyStart, dailyEnd, interval)?
=> Currently it can only be achieved by configuring multiple browser schedules with the same job class in different job names like this:
jobs:
- name: TestJob1
impl: xxx.yyy.zzz.JobClass
dailyStart: "03:00:00"
dailyEnd: "05:00:00"
interval: 10
- name: TestJob2
impl: xxx.yyy.zzz.JobClass
dailyStart: "15:00:00"
dailyEnd: "17:00:00"
interval: 10
The caveat of this approach is that two instances of the same job will be created.
- I can't see the optional GUI or browser on screen even with
gui.enable
,browser.type
,browser.headless
configured correctly when using docker image to run.
=> Before creating docker container, you may need to grant access control for X server by executingxhost +local:
, please see the manual ofxhost
for details. When running container, please add the environment variableDISPLAY
with proper value according to your environment and the mapping of volume path/tmp/.X11-unix
to run container, e.g.,docker run -d -e DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix kquiet/browser-scheduler:latest
. - Error:
The path to the driver executable must be set by the webdriver.xxxx.driver system property
is shown when I executed by java command directly.
=> An argument:-Dwebdriver.xxxx.driver=/path/to/driver/file
is required when executingjava
to make driver accessible. This argument is not required when executing in provided docker image as the drivers are located in path.