RGW Client

An Ceph RADOS Object Gateway client library, compatible with Amazon S3 API, supports bucket, object and admin operations.

Features:

• Amazon S3 v1/v2 sdk
• Support object, bucket and admin operations
• Support subscribe object event notification
• Asynchronous and listener callback
• Fluent unified APIs provided for easy assembling request
• Allow custom ActionExecutor to intercept actions and listeners
• Load balancing for multiple endpoints
• Fault tolerance
• Hide complexity of writing small or large object
• Concurrent writing
• Non blocking io and zero copy
• Reuse direct byte buffer
• Support Spring Boot
• Trace timing data of actions

Prerequisites

Install a Ceph storage cluster and object gateway. see Ceph Documentation for details.

Installation

1. Clone the repository from Github.

   git clone https://github.com/sosozhuang/rgw-client.git

2. Install maven packages.
   Since execute install will also run tests by default, change the cluster address and s3 credentials in properties files
   placed in test resources path before installing.

   mvn clean install

   Or, skip test cases with

   mvn -DskipTests=true clean install

Quick start

• Simple Java Application

1. Add dependency in Maven. Use aws v2 sdk:

   <dependency>
    <groupId>io.ceph</groupId>
    <artifactId>rgw-client-core-async</artifactId>
    <version>1.0.0</version>
    <type>pom</type>
   </dependency>

   or use aws v1 sdk:

   <dependency>
    <groupId>io.ceph</groupId>
    <artifactId>rgw-client-core-sync</artifactId>
    <version>1.0.0</version>
    <type>pom</type>
   </dependency>

2. Copy the content below into a properties file, replace endpoint, accessKey and secretKey with actual values.

   rgwclient.application.name = example
   rgwclient.connector.storages[0].endpoint = 192.168.100.3:8080
   rgwclient.connector.storages[0].accessKey = access_key
   rgwclient.connector.storages[0].secretKey = secret_key

   Properties file can come from jar files on your classpath.

3. Create Clients and execute an action.

   // load from file
   RGWClientProperties properties = RGWClientProperties.loadFromFile("rgwclient.properties");
   // load from class path
   // RGWClientProperties properties = RGWClientProperties.loadFromResource("rgwclient.properties");
   Clients clients = Clients.create(properties);
   ObjectClient objectClient = clients.getObject();
   ActionFuture<PutObjectResponse> future = objectClient.preparePutString()
    .withBucketName("bucket")
    .withKey("object")
    .withValue("hello world!")
    .execute();
   // wait until completed
   System.out.println(future.actionGet());

   More examples can be found in tests.

• Spring Boot Application

1. Add dependency in Maven.

   <dependency>
    <groupId>io.ceph</groupId>
    <artifactId>rgw-client-core-async</artifactId>
    <version>1.0.0</version>
    <type>pom</type>
   </dependency>
   <dependency>
    <groupId>io.ceph</groupId>
    <artifactId>rgw-client-spring-boot-starter</artifactId>
    <version>1.0.0</version>
   </dependency>

2. Copy the content below into application.properties, replace endpoint, accessKey and secretKey with actual values.

   spring.application.name = example
   rgwclient.connector.storages[0].endpoint = 192.168.100.3:8080
   rgwclient.connector.storages[0].accessKey = access_key
   rgwclient.connector.storages[0].secretKey = secret_key

3. Inject ObjectClient or AdminClient, BucketClient, SubscribeClient and execute an action.
   ```java
   @Autowired
   ObjectClient objectClient;

GetStringResponse response = objectClient.prepareGetString()
.withBucketName(“bucket”)
.withKey(“object”)
.run();
System.out.println(response.getContent());

### Examples
* Fluent API
```java
objectClient.preparePutString()
    .withBucketName("bucket")
    .withKey("key")
    .withValue("value")
    .withMetadata()
    .withContentEncoding("utf-8")
    .withContentType("text/plain")
    .endMetadata()
    .withACL()
    .withOwner("user", "name")
    .addGrant()
    .withPermission(Permission.READ_ACP)
    .endGrant()
    .endACL()
    .run()

• Asynchronous processing

ActionFuture<GetStringResponse> future = objectClient.preparePutString()
    .withBucketName("bucket")
    .withKey("key")
    .withValue("value")
    .execute()
System.out.println(future.actionGet());

• Callback listener

objectClient.preparePutString()
    .withBucketName("bucket")
    .withKey("key")
    .withValue("value")
    .execute(new ActionListener<PutObjectResponse>() {
        @Override
        public void onSuccess(PutObjectResponse response) {
            System.out.println(response);
        }
        @Override
        public void onFailure(Throwable cause) {
            cause.printStackTrace();
        }
    });

• ObjectWriter

ObjectWriter extends ObjectClient to support writing small or large object concurrently.
It can also write multiple string, byte array, byte buffer, and file into one object.

ObjectWriter writer = new ObjectWriter.Builder(objectClient)
    .withBuffer()
    .withMultiWriteThread()
    .withBucketName("bucket")
    .withKey("key")
    .build();
for (int i = 0; i < 10; i++) {
    new Thread(() -> writer.write("some string")).start();
}
BasePutObjectResponse response = writer.complete();
System.out.println(response);

More examples can be found in FileObjectWriterTest,
ByteBufMultiWriterTest,
ByteBufSingleWriterTest.

• Load balancing

If there are more than one endpoints, client side load balancing is enabled automatically.
For example:

spring.application.name = example
rgwclient.connector.storages[0].endpoint = 192.168.100.3:8080
rgwclient.connector.storages[0].accessKey = access_key
rgwclient.connector.storages[0].secretKey = secret_key
rgwclient.connector.storages[1].endpoint = 192.168.100.4:8080
rgwclient.connector.storages[1].accessKey = access_key
rgwclient.connector.storages[1].secretKey = secret_key

For spring boot application, if includes netflix ribbon module, ribbon load balance strategy is used. See spring cloud ribbon for advanced configuration.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>

• Trace

For spring boot application, if includes zipkin brace module, injects a bean of Tracer, the following content can be added to properties file.

rgwclient.enableTrace=true

Configurations

Various properties can be specified inside the properties file(Spring Boot applications use application.properties). The full configurations are described below:

Notice

This library uses Unsafe API concurrency control.

License

Code is licensed under the Apache License 2.0.