Spring 多个微服务做定时任务，如何做负载

基本思路

多个微服务，其业务的逻辑是一样的，自然包括定时任务。负载均衡在执行的时候，到达某个节点以后，定时任务都会执行，可以控制的思路就是使用队列的方式去操作。
如下有两种思路：

1. 将负载均衡的定时任务，从原先的直接执行业务逻辑修改为先将业务逻辑请求到队列中，然后让空闲的微服务去队列中自动领取；
2. 将负载均衡的定时任务，加锁进行操作。

ShedLock

ShedLock就是一种巧妙使用锁的方式的，在GitHub中的地址：GitHub - lukas-krecan/ShedLock: Distributed lock for your scheduled tasks

ShedLock does one and only one thing. It makes sure your scheduled tasks are executed at most once at the same time. If a task is being executed on one node, it acquires a lock which prevents execution of the same task from another node (or thread). Please note, that if one task is already being executed on one node, execution on other nodes does not wait, it is simply skipped.

Currently, Spring scheduled tasks coordinated through Mongo, JDBC database, Redis, Hazelcast or ZooKeeper are supported. More scheduling and coordination mechanisms and expected in the future.

ShedLock is not a distributed scheduler
Please note that ShedLock is not and will never be full-fledged scheduler, it's just a lock. If you need a distributed scheduler, please use another project. ShedLock is designed to be used in situations where you have scheduled tasks that are not ready to be executed in parallel, but can be safely executed repeatedly. For example if the task is fetching records from a database, processing them and marking them as processed at the end without using any transaction. In such case ShedLock may be right for you.

By setting lockAtMostFor we make sure that the lock is released even if the node dies and by setting lockAtLeastFor we make sure it's not executed more than once in fifteen minutes. Please note that if the task takes longer than 15 minutes, it will be executed again.

个人理解，拿JDBCTemplate进行说明，就是当第一个微服务执行定时任务的时候，会将此定时任务进行锁操作，然后其他的定时任务就不会再执行，锁操作有一定的时长，超过这个时长以后，再一次，所有的定时任务进行争抢下一个定时任务的执行权利，如此循环。其中两个配置lockAtMostFor和lockAtLeastFor，保证了在一个定时任务的区间内只有一个定时任务在执行，同时也保证了即便是其中的一个定时任务挂掉了，到一定的时间以后，锁也会释放，其他的定时任务依旧会进行执行权的争夺，执行定时任务。

使用步骤

1. 环境支持
   Java 8 、slf4j-api、Spring Framework(可选)

2. 依赖引入

<dependency>
  <groupId>net.javacrumbs.shedlock</groupId>
  <artifactId>shedlock-spring</artifactId>
  <version>0.18.2</version>
</dependency>

3. 定时任务的写法

import net.javacrumbs.shedlock.core.SchedulerLock;

...

@Scheduled(...)
@SchedulerLock(name = "scheduledTaskName")
public void scheduledTask() {
   // do something
}

The @SchedulerLock annotation has several purposes. First of all, only annotated methods are locked, the library ignores all other scheduled tasks. You also have to specify the name for the lock. Only one tasks with the same name can be executed at the same time.

You can also set lockAtMostFor attribute which specifies how long the lock should be kept in case the executing node dies. This is just a fallback, under normal circumstances the lock is released as soon the tasks finishes.

Lastly, you can set lockAtLeastFor attribute which specifies minimum amount of time for which the lock should be kept. Its main purpose is to prevent execution from multiple nodes in case of really short tasks and clock difference between the nodes.

• 示例
  Let's say you have a task which you execute every 15 minutes and which usually takes few minutes to run. Moreover, you want to execute it at most once per 15 minutes. In such case, you can configure it like this

import net.javacrumbs.shedlock.core.SchedulerLock;

...
private static final int FOURTEEN_MIN = 14 * 60 * 1000;
...

@Scheduled(cron = "0 */15 * * * *")
@SchedulerLock(name = "scheduledTaskName", lockAtMostFor = FOURTEEN_MIN, lockAtLeastFor = FOURTEEN_MIN)
public void scheduledTask() {
   // do something
}

By setting lockAtMostFor we make sure that the lock is released even if the node dies and by setting lockAtLeastFor we make sure it's not executed more than once in fifteen minutes. Please note that if the task takes longer than 15 minutes, it will be executed again.

4. 配置task
   Now we need to integrate the library into Spring. It's done by wrapping standard Spring task scheduler.

import net.javacrumbs.shedlock.spring.SpringLockableTaskSchedulerFactory;

...
@Bean
public ScheduledLockConfiguration taskScheduler(LockProvider lockProvider) {
    return ScheduledLockConfigurationBuilder
        .withLockProvider(lockProvider)
        .withPoolSize(10)
        .withDefaultLockAtMostFor(Duration.ofMinutes(10))
        .build();
}

Or if you already have an instance of ScheduledExecutorService

@Bean
public TaskScheduler taskScheduler(ScheduledExecutorService executorService, LockProvider lockProvider) {
    return SpringLockableTaskSchedulerFactory.newLockableTaskScheduler(executorService, lockProvider, Duration.of(10, MINUTES));
}

5. 配置锁
   锁的配置有多种方式的，如下的。

• Mongo

  • Import the project

  <dependency>
      <groupId>net.javacrumbs.shedlock</groupId>
      <artifactId>shedlock-provider-mongo</artifactId>
      <version>0.18.2</version>
  </dependency>
  

  • Configure:

  import net.javacrumbs.shedlock.provider.mongo.MongoLockProvider;
  
  ...
  
  @Bean
  public LockProvider lockProvider(MongoClient mongo) {
      return new MongoLockProvider(mongo, "databaseName");
  }
  

  Please note that MongoDB integration requires Mongo >= 2.4 and mongo-java-driver >= 3.4.0

• JdbcTemplate

  • Create the table

  CREATE TABLE shedlock(
    name VARCHAR(64), 
    lock_until TIMESTAMP(3) NULL, 
    locked_at TIMESTAMP(3) NULL, 
    locked_by  VARCHAR(255), 
    PRIMARY KEY (name)
  ) 
  

  • script for MS SQL is here

  CREATE TABLE [dbo].[shedlock](
    [name] [varchar](64) NOT NULL,
    [lock_until] [datetime] NULL,
    [locked_at] [datetime] NULL,
    [locked_by] [varchar](255) NOT NULL,
   CONSTRAINT [PK_shedlock] PRIMARY KEY CLUSTERED 
  (
    [name] ASC
  )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
  ) ON [PRIMARY]
  

  • Add dependency

  <dependency>
      <groupId>net.javacrumbs.shedlock</groupId>
      <artifactId>shedlock-provider-jdbc-template</artifactId>
      <version>0.18.2</version>
  </dependency>
  

  • Configure:

  import net.javacrumbs.shedlock.provider.jdbctemplate.JdbcTemplateLockProvider;
  
  ...
  
  @Bean
  public LockProvider lockProvider(DataSource dataSource) {
      return new JdbcTemplateLockProvider(dataSource);
  }
  

  Tested with MySql, Postgres and HSQLDB

• Plain JDBC

  • For those who do not want to use jdbc-template, there is plain JDBC lock provider. Just import

  <dependency>
      <groupId>net.javacrumbs.shedlock</groupId>
      <artifactId>shedlock-provider-jdbc</artifactId>
      <version>0.18.2</version>
  </dependency>
  

  • and configure

  import net.javacrumbs.shedlock.provider.jdbc.JdbcLockProvider;
  
  ...
  
  @Bean
  public LockProvider lockProvider(DataSource dataSource) {
      return new JdbcLockProvider(dataSource);
  }
  

  the rest is the same as with JdbcTemplate lock provider.

  • Warning
    Do not manually delete lock row or document from DB table or Mongo collection. ShedLock has an in-memory cache of existing locks so the row will NOT be automatically recreated until application restart. If you need to, you can edit the row/document, risking only that multiple locks will be held. Since 0.18.2 you can clean the cache by calling clearCache() on LockProvider.

• ZooKeeper (using Curator)

  • Import

  <dependency>
      <groupId>net.javacrumbs.shedlock</groupId>
      <artifactId>shedlock-provider-zookeeper-curator</artifactId>
      <version>0.18.2</version>
  </dependency>
  

  • and configure

  import net.javacrumbs.shedlock.provider.zookeeper.curator.ZookeeperCuratorLockProvider;
  
  ...
  
  @Bean
  public LockProvider lockProvider(org.apache.curator.framework.CuratorFramework client) {
      return new ZookeeperCuratorLockProvider(client);
  }
  

  By default, ephemeral nodes for locks will be created under /shedlock node.

• Redis (using Spring RedisConnectionFactory)

  • Import

  <dependency>
      <groupId>net.javacrumbs.shedlock</groupId>
      <artifactId>shedlock-provider-redis-spring</artifactId>
      <version>0.18.2</version>
  </dependency>
  

  • and configure

  import net.javacrumbs.shedlock.provider.redis.spring.RedisLockProvider;
  
  ...
  
  @Bean
  public LockProvider lockProvider(JedisPool jedisPool) {
      return new RedisLockProvider(connectionFactory, ENV);
  }
  

• Redis (using Jedis)

  • Import

  <dependency>
      <groupId>net.javacrumbs.shedlock</groupId>
      <artifactId>shedlock-provider-redis-jedis</artifactId>
      <version>0.18.0</version>
  </dependency>
  

  • and configure

  import net.javacrumbs.shedlock.provider.redis.jedis.JedisLockProvider;
  
  ...
  
  @Bean
  public LockProvider lockProvider(JedisPool jedisPool) {
      return new JedisLockProvider(jedisPool, ENV);
  }
  

• Hazelcast

  • Import the project

  <dependency>
      <groupId>net.javacrumbs.shedlock</groupId>
      <artifactId>shedlock-provider-hazelcast</artifactId>
      <version>0.18.0</version>
  </dependency>
  

  • Configure:

  import net.javacrumbs.shedlock.provider.hazelcast.HazelcastLockProvider;
  
  ...
  
  @Bean
  public HazelcastLockProvider lockProvider(HazelcastInstance hazelcastInstance) {
      return new HazelcastLockProvider(hazelcastInstance);
  }
  

• Spring XML configuration

  • If you are using Spring XML config, use this configuration

  <!-- lock provider of your choice (jdbc/zookeeper/mongo/whatever) -->
  <bean id="lockProvider" class="net.javacrumbs.shedlock.provider.jdbctemplate.JdbcTemplateLockProvider">
      <constructor-arg ref="dataSource"/>
  </bean>
  
  <bean id="scheduler" class="net.javacrumbs.shedlock.spring.SpringLockableTaskSchedulerFactoryBean">
      <constructor-arg>
          <task:scheduler id="sch" pool-size="10"/>
      </constructor-arg>
      <constructor-arg ref="lockProvider"/>
      <constructor-arg name="defaultLockAtMostFor">
          <bean class="java.time.Duration" factory-method="ofMinutes">
              <constructor-arg value="10"/>
          </bean>
      </constructor-arg>
  </bean>
  
  
  <!-- Your task(s) without change (or annotated with @Scheduled)-->
  <task:scheduled-tasks scheduler="scheduler">
      <task:scheduled ref="task" method="run" fixed-delay="1" fixed-rate="1"/>
  </task:scheduled-tasks>
  

  • Annotate scheduler method(s)

  @SchedulerLock(name = "taskName")
  public void run() {
  
  }
  

• Running without Spring

  • It is possible to use ShedLock without Spring

  LockingTaskExecutor executor = new DefaultLockingTaskExecutor(lockProvider);
  
  ...
  
  Instant lockAtMostUntil = Instant.now().plusSeconds(600);
  executor.executeWithLock(runnable, new LockConfiguration("lockName", lockAtMostUntil));