Scripts đź”—

All datasets link to a scripts catalog where the index of automation scripts that have been successfully executed are listed.

{
   "element": "shoji:entity",
   "self": "https://app.crunch.io/api/datasets/123456/",
   "body": {
      "name": "My new project",
      //...
   },
   "catalogs: {
      // ...
      "scripts": "https://app.crunch.io/api/datasets/123456/scripts/"
      // ...
   }
}
GET /api/datasets/123456/scripts/ HTTP/1.1

{
    "element": "shoji:catalog",
    "self": "https://app.crunch.io/api/datasets/123456/scripts/",
    "index": {}
}

Creating and executing scripts đź”—

By default all datasets start with an empty script’s catalog. Creating new scripts via POST will populate them.

POST /api/datasets/123456/scripts/ HTTP/1.1

{
    "element": "shoji:entity",
    "body" {
        "body": "<Script contents>"
    }
}

The server will perform a synchronous validation of the script’s contents and consistency against the current dataset’s schema. In case of error, the server will return a 400 error response with a crunch:error as the body containing the list of errors.

{
    "element": "crunch:error",
    "self": "https://app.crunch.io/api/datasets/123456/scripts/",
    "type": "scripts:syntax",
    "description": "Errors processing the script",
    "resolutions": [
       {
           "line": 0,
           "line": 0,
           "command": 0,
           "message": ""
       },
       {
           "line": 0,
           "line": 0,
           "command": 0,
           "message": ""
       },
       // ...
    ]
}

The errors will be under the resolutions key containing a list of errors descriptions indicating the command number, and script line and column where for each error.

In the case of a successful validation, the server will return a 202 response with a shoji:view response body pointing to a progress indicator to follow the script’s execution status.

The dataset remains locked for changes during this script execution period.

On success, the new script entity will be created and available from the scripts catalog.

{
  "element": "shoji:entity",
  "self": "https://app.crunch.io/api/datasets/123456/scripts/qwerty/",
  "body": {
    "body": "...",
    "items_created": 2,
    "creation_time": "2020-07-08T22:49:50.000000+00:00",
    "mutations": true
  },
  "catalogs": {
    "output": "https://app.crunch.io/api/datasets/123456/scripts/qwerty/output/"
  },
  "views": {
    "savepoint": "https://app.crunch.io/api/datasets/123456/savepoints/abc/"
  },
  "fragments": {
    "revert": "https://app.crunch.io/api/datasets/123456/scripts/qwerty/revert/"
  }
}

Reverting scripts đź”—

There are two ways of reverting the output of a script. The “Soft” delete or “undo” of a script’s created artifacts and variables, or a hard revert that will bring back the dataset to the state it was before running such script.

The difference between both is that reverting will restore the dataset dropping all following scripts and their output (artifacts and variables), while an undo will only delete the artifacts and variables created by this script, but changes done by other scripts and this script’s record will remain in place.

Undo đź”—

Undoing a script’s output is done by performing a DELETE request on the script’s output catalog.

In case that there are any dependencies that prevent the artifacts from being deleted the request will return a 409 response.

DELETE /api/datasets/123456/scripts/output/ HTTP/1.1

Restore đź”—

The application stores a savepoint of the dataset directly before the execution of the commands in a script.

To perform a script driven restore where all the associated non versioned artifacts are deleted, the client must perform a POST request to the revert endpoint. This will fire up an asynchronous task to iteratively drop all the artifacts from all scripts and restore the dataset.

In case that there are any dependencies that prevent this script from being reverted the request will return a 409 response.

POST /api/datasets/123456/scripts/revert/ HTTP/1.1