get ( "some_key" ) # utilizing a pool redis_pool. # start monitor script $ docker exec -it a0eb6d2f82a7 /app/monitor.# original style, which is deprecated and would block across threads Redis. # start containers $ docker-compose build & docker-compose -compatibility upĬONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESĥ5bb702657fc redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_9Ĩbdbb033e247 redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_6Ģ56a165d1897 redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_7ī5f2256d839b redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_2ĩd6b6646c481 redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_8Ħ8b4fd1ac5e3 redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_4ġ85da4e137df redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_10Ī0eb6d2f82a7 redis-worker_producer "/app/producer.rb" 3 minutes ago Up 3 minutes redis-worker_producer_1ĭ0b18174daa6 redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_1ĩ405439235da redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_3ĩ904f7ab16af redis-worker_worker "/app/worker.rb" 3 minutes ago Up 3 minutes redis-worker_worker_5Ħ57d2386da40 redis:latest "docker-entrypoint.s…" 3 minutes ago Up 3 minutes 0.0.0.0:6379->6379/tcp redis-worker_redis_1 I started the docker containers via compose and then executed the monitor script inside the producer container to show the results. dockerfile : Dockerfile command : /app/worker.rb deploy : mode : replicated replicas : 10 environment : - REDIS_HOST=redis depends_on : - redis dockerfile : Dockerfile command : /app/producer.rb environment : - REDIS_HOST=redis depends_on : - redis worker : build : context. new file: docker-compose.yml version : ' 3' services : redis : image : redis:latest ports : - " 6379:6379" producer : build : context. I defined a docker compose file to start Redis, a producer, and 10 workers. new file: monitor.rb #!/usr/bin/env ruby require_relative 'redis_base' class RedisClient ~/.gemrc I created a monitor script to show queued tasks and the completed tasks for each worker. new file: producer.rb #!/usr/bin/env ruby require_relative 'redis_base' class RedisProducer e # TODO: put task back in work queue log :error, e end end RedisWorker. In a loop, it queues work by pushing a task into the work queue, and then publishes to the pub/sub channel to inform subscribers there is new work to complete. I defined the producer class (RedisProducer) below. lrange, 0, - 1 end def log ( level, message ). ![]() new ( host:, port: ) end protected def queue_list. fetch ( 'REDIS_HOST', 'localhost' ) = ENV. fetch ( 'WORK_CHANNEL', 'work_channel' ) = ENV. ![]() fetch ( 'WORK_QUEUE', 'work_queue' ) = ENV. new file: redis_base.rb require 'json' require 'logger' require 'redis' require 'securerandom' class RedisBase def initialize = SecureRandom. On initialize it creates a connection to Redis from environment variables. It contains all the Redis client methods. I first defined a RedisBase parent class that the workers and producer with inherit from. One list will contain strings of the task name to complete, and another will contain a JSON string of the completed task and the worker that completed it. In this post I’ll share some Ruby code that uses Redis Pub/Sub and Redis lists to implement work queues.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |