python>> cdap-maven-plugin>> 返回
项目作者： cdapio

项目描述 ：
Collection of CDAP Maven Plugins
高级语言： Java
项目地址: git://github.com/cdapio/cdap-maven-plugin.git
创建时间： 2017-03-19T10:00:51Z
项目社区：https://github.com/cdapio/cdap-maven-plugin
开源协议：Apache License 2.0
下载


CDAP Maven Plugin

Join CDAP community License

This Maven Plugin has 4 goals:

  • mvn cdap:create-plugin-json
    • Generates a CDAP Plugin JSON file
  • mvn cdap:create-plugin-spec-json
    • Generates a spec.json file
  • mvn cdap:package-artifacts-for-hub
    • Moves artifacts to a user specified output dir
    • Note: cannot be run alone
  • mvn cdap:create-pipeline-json
    • Extracts configuration from exported pipelines
    • Suitable for Hub

Maven Goals

  1. Create Plugin Json
  2. Create Plugin Spec JSON
  3. Move Artifact Files
  4. Extract Configuration From Exported Pipelines

Create Plugin Json

This goal generates the cdap plugin json file. The maven goal is create-plugin-json.

To run only the goal use:

mvn cdap:create-plugin-json

The generated file will be placed in the ${project.build} directory.

Configurations

Configuration Required Default Description
cdapArtifacts Y N/A Specifies all the parent CDAP artifacts and scope this plugin is applicable for.
widgetsDirectory N ${project.dir}/widgets Specifies alternate widgets directory.
docsDirectory N ${project.dir}/docs Specifies alternate documentation directory.
iconsDirectory N ${project.dir}/icons Specifies alternate icons directory.

Example POM

  1. <plugin>
  2.   <groupId>co.cask</groupId>
  3.   <version>1.1.3</version>
  4.   <artifactId>cdap-maven-plugin</artifactId>
  5.   <configuration>
  6.     <cdapArtifacts>
  7.       <parent>system:cdap-data-pipeline[4.0.0,9.0.0-SNAPSHOT)</parent>
  8.       <parent>system:cdap-data-streams[4.0.0,9.0.0-SNAPSHOT)</parent>
  9.     </cdapArtifacts>
  10.   </configuration>
  11.   <executions>
  12.     <execution>
  13.        <id>create-artifact-config</id>
  14.        <phase>prepare-package</phase>
  15.        <goals>
  16.          <goal>create-plugin-json</goal>
  17.        </goals>
  18.     </execution>
  19.   </executions>
  20. </plugin>

Output

  1. [INFO] ------------------------------------------------------------------------
  2. [INFO] Building Trash Sink 1.1.0-SNAPSHOT
  3. [INFO] ------------------------------------------------------------------------
  4. [INFO] 
  5. [INFO] --- cdap-maven-plugin:1.0-SNAPSHOT:create-plugin-json (default-cli) @ trash-plugin ---
  6. [INFO] ------------------------------------------------------------------------
  7. [INFO] CDAP Plugin JSON
  8. [INFO] ------------------------------------------------------------------------
  9. [INFO] Project              : Trash Sink
  10. [INFO] Group ID             : co.cask
  11. [INFO] Artifact ID          : trash-plugin
  12. [INFO] Version              : 1.1.0-SNAPSHOT
  13. [INFO] Base Directory       : /Users/nitin/Work/Devel/plugin-json-test
  14. [INFO] Build Directory      : /Users/nitin/Work/Devel/plugin-json-test/target
  15. [INFO] Widgets Directory    : /Users/nitin/Work/Devel/plugin-json-test/widgets
  16. [INFO] Docs Directory       : /Users/nitin/Work/Devel/plugin-json-test/docs
  17. [INFO] CDAP Artifacts
  18. [INFO]  system:cdap-data-pipeline[4.0.0,9.0.0-SNAPSHOT)
  19. [INFO]  system:cdap-data-streams[4.0.0,9.0.0-SNAPSHOT)
  20. [INFO] ------------------------------------------------------------------------
  21. [INFO] Successfully created  : trash-plugin-1.1.0-SNAPSHOT.json
  22. [INFO] ------------------------------------------------------------------------
  23. [INFO] ------------------------------------------------------------------------
  24. [INFO] BUILD SUCCESS
  25. [INFO] ------------------------------------------------------------------------
  26. [INFO] Total time: 1.026 s
  27. [INFO] Finished at: 2017-03-19T03:21:38-07:00
  28. [INFO] Final Memory: 13M/309M
  29. [INFO] ------------------------------------------------------------------------

Create Plugin Spec JSON

This goal generates the spec.json file. The maven goal is create-plugin-spec-json

To run only the goal use:

mvn cdap:create-plugin-spec-json

The generated file will be placed in the ${project.build} directory.

Configuration

Configuration Required Default Description
isPipeline N false Specifies if Spec.json is for a pipeline.
actionType N one_step_deploy_plugin Specifies actionType - needs to be create_pipeline_draft if isPipeline true.
scope N user Artifact scope.
version N ${project.version} Artifact version.
cdapArtifacts Y N/A Specifies all the parent CDAP artifacts and scope this plugin is applicable for.
label N ${project.name} Short description that will be displayed to users during the install process.
actionType Y one_step_deploy_plugin Action specification type.
description N ${project.description} Plugin description.
author N ${project.author} Plugin author.
org N ${project.org} Plugin author organisation.
categories N N/A List of Categories.
buildDirectory N ${project.build.directory} Target directory for the spec.json file.
artifactId N ${project.artifactId} Used to reference the config (json) file name.
additionalActionArguments N N/A List of key/value parameters to include in the Action.

NOTE: Categories affect the display of the HUB - please include at the bare minimum the team and type of item

Example POM

Including both create-plugin-json and create-plugin-spec-json goals

  1. <plugin>
  2.   <groupId>co.cask</groupId>
  3.   <version>1.1.3</version>
  4.   <artifactId>cdap-maven-plugin</artifactId>
  5.   <configuration>
  6.     <cdapArtifacts>
  7.       <parent>system:cdap-data-pipeline[6.0.0-SNAPSHOT,9.0.0-SNAPSHOT)</parent>
  8.       <parent>system:cdap-data-streams[6.0.0-SNAPSHOT,9.0.0-SNAPSHOT)</parent>
  9.     </cdapArtifacts>
  10.     <author>LiveRamp</author>
  11.     <org>LiveRamp</org>
  12.     <description>optional plugin description</description>
  13.     <categories>
  14.       <category>cat1</category>
  15.       <category>cat2</category>
  16.     </categories>
  17.     <additionalActionArguments>
  18.       <argument>
  19.         <key>jar</key>
  20.         <value>${project.artifactId}-${project.version}.jar</value>
  21.       </argument>
  22.     </additionalActionArguments>
  23.     <scope>user</scope>
  24.   </configuration>
  25.   <executions>
  26.     <execution>
  27.       <id>create-artifact-config</id>
  28.       <phase>prepare-package</phase>
  29.       <goals>
  30.         <goal>create-plugin-json</goal>
  31.         <goal>create-plugin-spec-json</goal>
  32.       </goals>
  33.     </execution>
  34.   </executions>
  35. </plugin>

Including only create-plugin-spec-json goal

  1. <plugins>
  2.  <plugin>
  3.    <groupId>co.cask</groupId>
  4.    <artifactId>cdap-maven-plugin</artifactId>
  5.    <version>1.1.3</version>
  6.    <configuration>
  7.      <cdapArtifacts>
  8.        <parent>system:cdap-data-pipeline[6.0.0-SNAPSHOT,9.0.0-SNAPSHOT)</parent>
  9.        <parent>system:cdap-data-streams[6.0.0-SNAPSHOT,9.0.0-SNAPSHOT)</parent>
  10.      </cdapArtifacts>
  11.      <categories>
  12.          <param>cat1</param>
  13.      </categories>
  14.      <actionType>create_pipeline_draft</actionType>
  15.      <isPipeline>true</isPipeline>
  16.    </configuration>
  17.    <executions>
  18.      <execution>
  19.        <id>create-artifact-config</id>
  20.        <phase>prepare-package</phase>
  21.        <goals>
  22.          <goal>create-plugin-spec-json</goal>
  23.        </goals>
  24.      </execution>
  25.    </executions>
  26.  </plugin>
  27. </plugins>
  28. </build>
  30. </project>

Output

  1. [INFO] ------------------------------------------------------------------------
  2. [INFO] CDAP Plugin JSON
  3. [INFO] ------------------------------------------------------------------------
  4. [INFO] Project              : Adhoc SQL Plugin
  5. [INFO] Group ID             : co.cask
  6. [INFO] Artifact ID          : adhoc_sql
  7. [INFO] Version              : 0.1.0
  8. [INFO] Base Directory       : /Users/user1/code/cdap-plugins/lsh/misc/adhoc_sql
  9. [INFO] Build Directory      : /Users/user1/code/cdap-plugins/lsh/misc/adhoc_sql/target
  10. [INFO] Widgets Directory    : widgets
  11. [INFO] Icons Directory      : icons
  12. [INFO] Docs Directory       : docs
  13. [INFO] CDAP Artifacts
  14. [INFO]  system:cdap-data-pipeline[6.0.0-SNAPSHOT,9.0.0-SNAPSHOT)
  15. [INFO]  system:cdap-data-streams[6.0.0-SNAPSHOT,9.0.0-SNAPSHOT)
  16. [INFO] ------------------------------------------------------------------------
  17. [INFO] Successfully created  : adhoc_sql-0.1.0.json
  18. [INFO] ------------------------------------------------------------------------
  19. [INFO]
  20. [INFO] --- cdap-maven-plugin:1.1.1-SNAPSHOT:create-plugin-spec-json (create-artifact-config) @ adhoc_sql ---
  21. [INFO] ------------------------------------------------------------------------
  22. [INFO] CDAP Plugin JSON - Spec Json creator
  23. [INFO] ------------------------------------------------------------------------
  24. [INFO] specVersion          : 0.1.0
  25. [INFO] label                : Adhoc SQL Plugin
  26. [INFO] description          : optional plugin description
  27. [INFO] author               : LiveRamp
  28. [INFO] org                  : LiveRamp
  29. [INFO] created              : 1608720834834
  30. [INFO] cdapVersion          : [6.0.0-SNAPSHOT,9.0.0-SNAPSHOT)
  31. [INFO] action type          : one_step_deploy_plugin
  32. [INFO] action label         : Adhoc SQL Plugin
  33. [INFO] action arg name      : Adhoc SQL Plugin
  34. [INFO] action arg version   : 0.1.0
  35. [INFO] action arg scope     : user
  36. [INFO] action arg config    : adhoc_sql-0.1.0.json
  37. [INFO] action arg jar       : adhoc_sql-0.1.0.jar
  38. [INFO] categories
  39. [INFO]  cat1
  40. [INFO]  cat2
  41. [INFO] ------------------------------------------------------------------------
  42. [INFO] Successfully created: spec.json
  43. [INFO] ------------------------------------------------------------------------

Move Artifact Files

This plugin aims to move built artifacts packages to a common user specified output directory in the format expected
by a CDAP HUB and HUB packaging/publishing tools.
The goal is useful in case you’d want to run CDAP publisher from that output directory.

The maven goal is package-artifacts-for-hub

Configuration

Configuration Required Default Description
relativeOutputDir Y “${project.build.directory}/packages/“ Specifies the Output Directory Relative to the module build directory

Example relativeOutputDir value : /../../../../packages/

Using Directory Maven Plugin

To simplify the relativeOutputDir value the directory-maven-plugin should be used in the parent Pom

Example of directory-maven-plugin plugin configuration to have in parent pom.xml

  1. <plugin>
  2.   <groupId>org.commonjava.maven.plugins</groupId>
  3.   <artifactId>directory-maven-plugin</artifactId>
  4.   <version>0.3.1</version>
  5.   <executions>
  6.     <execution>
  7.       <id>directories</id>
  8.       <goals>
  9.         <goal>directory-of</goal>
  10.       </goals>
  11.       <phase>initialize</phase>
  12.       <configuration>
  13.         <property>parent_module.basedir</property>
  14.         <project>
  15.           <groupId>com.org.groupId</groupId>
  16.           <artifactId>artifactId</artifactId>
  17.         </project>
  18.       </configuration>
  19.     </execution>
  20.   </executions>
  21. </plugin>

Using this simplifies relativeOutputDir value to ${parent_module.basedir}/packages.

If left to default value the output will be placed under the module build dir

Execution settings

The goal must be run after any pre-package goals specified. In order to do this the following execution settings need
to be applied.

  1. <execution>
  2.   <goals>
  3.     <goal>package-artifacts-for-hub</goal>
  4.   </goals>
  5.   <id>move-artifacts</id>
  6.   <phase>package</phase>
  7. </execution>

Example POM

  1. <plugin>
  2.   <groupId>co.cask</groupId>
  3.   <version>1.1.3</version>
  4.   <artifactId>cdap-maven-plugin</artifactId>
  5.   <configuration>
  6.     <cdapArtifacts>
  7.       <parent>system:cdap-data-pipeline[6.1.0-SNAPSHOT,7.0.0-SNAPSHOT)</parent>
  8.       <parent>system:cdap-data-streams[6.1.0-SNAPSHOT,7.0.0-SNAPSHOT)</parent>
  9.     </cdapArtifacts>
  10.     <additionalActionArguments>
  11.       <argument>
  12.         <key>jar</key>
  13.         <value>${project.artifactId}-${project.version}.jar</value>
  14.       </argument>
  15.     </additionalActionArguments>
  16.     <scope>user</scope>
  17.     <author>LiveRamp</author>
  18.     <org>LiveRamp</org>
  19.     <description>optional plugin description</description>
  20.     <!-- Using directory maven plugin -->
  21.     <relativeOutputDir>${parent_module.basedir}/packages</relativeOutputDir>
  22.   </configuration>
  23.   <executions>
  24.     <execution>
  25.       <goals>
  26.         <goal>create-plugin-json</goal>
  27.         <goal>create-plugin-spec-json</goal>
  28.       </goals>
  29.       <id>create-artifact-config</id>
  30.       <phase>prepare-package</phase>
  31.     </execution>
  32.     <execution>
  33.       <goals>
  34.         <goal>package-artifacts-for-hub</goal>
  35.       </goals>
  36.       <id>move-artifacts</id>
  37.       <phase>package</phase>
  38.     </execution>
  39.   </executions>
  40. </plugin>

Output

  1. [INFO] 
  2. [INFO] --- cdap-maven-plugin:1.1.2-SNAPSHOT:package-artifacts-for-hub (move-artifacts) @ adhoc_sql ---
  3. [INFO] Creating artifact output dir at: /Users/user1/code/cdap-plugins/lsh/misc/adhoc_sql/target/../../../../packages/adhoc_sql/0.1.0
  4. [INFO] Copying adhoc_sql-0.1.0.jar to /Users/user1/code/cdap-plugins/lsh/misc/adhoc_sql/target/../../../../packages/adhoc_sql/0.1.0/adhoc_sql-0.1.0.jar
  5. [INFO] Copying adhoc_sql-0.1.0.json to /Users/user1/code/cdap-plugins/lsh/misc/adhoc_sql/target/../../../../packages/adhoc_sql/0.1.0/adhoc_sql-0.1.0.json
  6. [INFO] Copying spec.json to /Users/user1/code/cdap-plugins/lsh/misc/adhoc_sql/target/../../../../packages/adhoc_sql/0.1.0/spec.json

Packages moved to output directory:

  1. - packages
  2.   - artifact-A
  3.     - 1.0.0
  4.       - artifact-A-1.0.0.jar 
  5.       - artifact-A-1.0.0.json
  6.       - spec.json
  7.   - artifact-B
  8.     - 1.1.0
  9.       - artifact-B-1.1.0.jar 
  10.       - artifact-B-1.1.0.json
  11.       - spec.json
  12.       - icon.json [Optional]

Extract Configuration From Exported Pipelines

Maven goal to extract and rename an exported Pipeline JSON so that it is able to be published and consumed by a HUB

The goal is create-pipeline-json

Configuration

Configuration Required Default Description
pipelineJsonFile Y “” Specifies the exported Json file

Note: The pipelineJsonFile value needs to be the full path hence using ${project.baseDir} will be needed.

E.g: ${project.basedir}/exported_jsonFile.json

If the path is not correct you will see a

  1. [ERROR] exported_jsonFile.json

Example POM

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
  3.          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  5.   <name>Test Exported Pipeline</name>
  6.   <groupId>co.cask</groupId>
  7.   <artifactId>test_exported_pipeline</artifactId>
  8.   <version>1.0.0</version>
  9.   <modelVersion>4.0.0</modelVersion>
  11.   <properties>
  12.     <main.basedir>${project.basedir}</main.basedir>
  13.     <!-- properties for script build step that creates the config files for the artifacts -->
  14.     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  15.   </properties>
  17.   <build>
  18.     <plugins>
  19.       <plugin>
  20.         <groupId>co.cask</groupId>
  21.         <version>1.1.3</version>
  22.         <artifactId>cdap-maven-plugin</artifactId>
  23.         <configuration>
  24.           <pipelineJsonFile>${basedir}/../exported_pipeline.json</pipelineJsonFile>
  25.         </configuration>
  26.         <executions>
  27.           <execution>
  28.             <id>create-artifact-config</id>
  29.             <phase>prepare-package</phase>
  30.             <goals>
  31.               <goal>create-pipeline-json</goal>
  32.             </goals>
  33.           </execution>
  34.         </executions>
  35.       </plugin>
  36.     </plugins>
  37.   </build>
  38. </project>

Output

A exported_pipeline.json file should be created in the specified path in pipelineJsonFile property.