项目作者: malayh

项目描述 :
Transient inmemory Messaging Queue
高级语言: Python
项目地址: git://github.com/malayh/tinyMQ.git
创建时间: 2021-04-03T09:03:55Z
项目社区:https://github.com/malayh/tinyMQ

开源协议:

下载


tinyMQ

It is a transient inmemory messaging queue for systems where you don’t have enought resouces to use Kafka or RabbitMQ.

It uses ProtocolBuffers underneath to pass messages over persistent websockets. The API is simple and you can install and setup the server in less than 3 minutes.

Install

  • clone this repo
  • Run python setup.py install with your virtualenv activated
  • Start the server with tmqserver --startserver --host localhost --port 9800
  • Stop the server with tmqserver --stopserver

Usage

The client API works using tinyMQ.Producer and tinyMQ.Consumer classes

Following is a simple Producer that writes “Hi” to a topic named test-topic

  1. from tinyMQ import Producer
  2. import asyncio
  3. async def producer(host: str, port: int, topic: str):
  4. producer = Producer(host, port,topic)
  5. await producer.init_conn()
  6. for i in range(100):
  7. await producer.send("Hi", i)
  8. await asyncio.sleep(1)
  9. async def main():
  10. await producer("localhost",9800,"test-topic")
  11. if __name__ == "__main__":
  12. asyncio.get_event_loop().run_until_complete(main())

Following is a exmple of a Consumer consuming from topic test-topic

  1. import asyncio
  2. from tinyMQ import Consumer
  3. async def consumer(host:str, port:int, topic: str):
  4. consumer = Consumer(host, port, topic)
  5. await consumer.init_conn()
  6. # if there is nothing to consume, sleep for 1 sec
  7. while True:
  8. msg = await consumer.poll()
  9. if not msg:
  10. await asyncio.sleep(1)
  11. continue
  12. print(f"Msg id: {msg[0]} Msg:{msg[1]}")
  13. async def main():
  14. await consumer("localhost",9800,"test-topic")
  15. if __name__ == "__main__":
  16. asyncio.get_event_loop().run_until_complete(main())

API

All the messages are stores inside topics. A topic is created whenever a Consumer or Producers is connected to tinyMQ with a topic name that does not exist yet. Messages will be deleted from the queue once it is consumed.

Client API

tinyMQ.Producer

This class is to be used to write messgess to the queue.

  • __init__(host: str, port: int , topic: str)
    • Initialize a connection as a Producer to a tinyMQ instance running on host and on port. It will write to a topic named topic which will be created if doenot exist already
  • await init_conn()
    • This has to be called to make a connection usable.
    • Throws ConnectionRefusedError if unable to connect
  • await send(message:str, id:int)
    • Writes message to the topic connected to with id id
    • Uniqueness of the messages send and recieved can be controlled by the users using the id that is sent with the message
    • Throws tinyMQ.DeadConnectionUsed if connection is closed by the server or if init_conn is not called

      tinyMQ.Consumer

      This class is used to consumed data from a topic. Only one consumer can consume from a given topic.
  • __init__(host: str, port: int , topic: str)
    • Initialize a connection as a Consumer to a tinyMQ instance running on host and on port. It will read from a topic named topic which will be created if doenot exist already
  • await init_conn()

    • This has to be called to make a connection usable.
    • Throws ConnectionRefusedError if unable to connect
    • Throws tinyMQ.InitFailed if the mentiond topic is already being consumed by some other consumer
  • await poll()

    • Checks if there is any message to be consumed.
    • Returns None if topic is empty. Otherwise returns tuple (id,message)
    • Throws tinyMQ.DeadConnectionUsed if connection is closed by the server