项目作者: Bogdaan

项目描述 :
Lightweight url shortner with key/value database
高级语言: Go
项目地址: git://github.com/Bogdaan/gurl.git
创建时间: 2017-10-17T17:33:58Z
项目社区:https://github.com/Bogdaan/gurl

开源协议:MIT License

下载


gurl

Hight perfomance and compact url shortening server.

Features

  1. +---------------------+
  2. | |
  3. | Gurl |
  4. | |
  5. | +---------------+ | +-----------------+
  6. | | | | | |
  7. | | api <-------+ You backend |
  8. | | | | | |
  9. | +---------------+ | +--------^--------+
  10. | | |
  11. | | |
  12. | +---------------+ | +--------+--------+ +-----------------+
  13. | | | | | | | |
  14. | | redirect <-------+ reverse proxy <------+ User |
  15. | | | | | | | |
  16. | +---------------+ | +-----------------+ +-----------------+
  17. | |
  18. +---------------------+

Why gurl?

You can find lots of solution at github but:

  1. It requires remote storage (like mysql or MongoDB) or operate with low-performance storage system (like sqlite, or own file storate).

  2. It depdends on Mysql or PostgreSQL features (like AUTO_INCREMENT etc.)

  3. It Mix api server and redirect server (same server handle both requests)

  4. It marshal / unmarshal JSON

  5. It contain complex logic, validation, huge data structures

Gurl devoid these disadvantages.

Internals

Gurl start two HTTP servers:

  • api server - manage links (crete, find, update etc.)
  • redirect server - just find link, and send 301 (or 404)
  1. $ gurl --help
  2. Usage of gurl:
  3. -api-address string
  4. Control server bind address (default ":7070")
  5. -database string
  6. Database file path (default "links.db")
  7. -redirect-address string
  8. Redirect server bind address (default ":8090")

Storage schema

Each short link represents a [12]byte array:

  1. 4 byte time preffix, with format “0601” (see time).
  2. 8 byte hash, created by xxhash and encoded in base36

Api server interface

POST link/add

Create one (or more) short urls and returns CSV with hashes. Arguments:

  • link - string (requred) - links for shortening, each in new line
  1. curl -X POST -d $'link=http://1.x\nhttp://2.x' http://localhost:7070/link/add
  2. http://1.x,17107z1g4rbg
  3. http://2.x,17101wa6cbnd

Find full link by hash. Arguments:

  • hash - string (requred)
  1. curl -X GET http://localhost:7070/link/byHash?hash=17101wa6cbnd
  2. http://2.x

Find list of hashes (part of sorted list) by arguments:

  • start - string (optional)
  • end - string (optional)
  1. curl -X GET 'http://localhost:7070/link/list?start=17101wa6cbnd&end=17102wa6cbnd'
  2. 17101wa6cbnd,http://2.x
  3. 171020y8fgzn,http://x2.x2
  4. 17102ewiejoz,http://xx.xx
  5. 17102msb5big,http://v.v
  6. # OR only start
  7. curl -X GET 'http://localhost:7070/link/list?start=17101wa6cbnd'

POST hash/remove

Remove hashes. Arguments:

  • hash - string (required) - one ore more hashes, each in new line
  1. curl -X POST -d 'hash=17101wa6cbnd' 'http://localhost:7070/hash/remove'
  2. 17101wa6cbnd

POST hash/cleanup

Remove hashes by range (probably day range). Arguments:

  • start (optional)
  • end (optional)
  1. curl -X POST -d 'start=171020y8fgzn&end=17102msb5big' 'http://localhost:7070/hash/cleanup'
  2. total,3

GET backup

Build consistent storage backup.

  1. curl -X GET 'http://localhost:7070/backup'
  2. ...cut

Test redirect server

  1. # Create short url for http://example.com
  2. curl -X POST -d 'link=http://example.com' 'http://localhost:7070/link/add'
  3. http://example.com,17105tai0tvo
  4. # Found hash "17105tai0tvo", send request to redirect server
  5. curl -v -X GET 'http://localhost:8090/17105tai0tvo'
  6. * Hostname was NOT found in DNS cache
  7. * Trying 127.0.0.1...
  8. * Connected to localhost (127.0.0.1) port 8090 (#0)
  9. > GET /17105tai0tvo HTTP/1.1
  10. > User-Agent: curl/7.35.0
  11. > Host: localhost:8090
  12. > Accept: */*
  13. >
  14. < HTTP/1.1 302 Found
  15. < Location: http://example.com
  16. < Date: Mon, 16 Oct 2017 12:05:01 GMT
  17. < Content-Length: 41
  18. < Content-Type: text/html; charset=utf-8
  19. <
  20. <a href="http://example.com">Found</a>.
  21. * Connection #0 to host localhost left intact

Pitfalls

  • Gurl is not scalable.
  • xxhash - high-quality hashing algorithm that is much faster than anything in the Go standard library
  • bolt - pure Go key/value store
  • gurl-cli - command line util for system admin