tainacan/docs/exporter-flow.md

# The Exporter flow

This page describes how Tainacan exporters work and is a reference to write your own exporter. 

This documentation is still in construction. A very effective way to learn more on how to build an exporter is to look at the source code of the CSV Exporter class that is included in the tainacan package.

## Introduction

Exporters can export items from a single collection, or even create a bunch of collections, taxonomies and items all at once.

In order to create an Exporter, you have to extend the `\Tainacan\Exporter` Class and register it using the global `Tainacan\Export_Handler->register_exporter()` method.

This method takes an array as argument, with the defintion of your exporter. These are the expected attributes.

```
	 	   @type string		 $name					The name of the exporter. e.g. 'Example exporter'
	 	   @type string		 $slug					A unique slug for the exporter. e.g. 'example-exporter'
	 	   @type string		 $description			The exporter description. e.g. 'This is an example exporter description'
	 	   @type string		 $class_name			The Exporter Class. e.g. '\Tainacan\Exporter\Test_Exporter'
	 	   @type bool		 $manual_mapping		Wether Tainacan must present the user with an interface to manually choose 
	 												a mapping standard. This will allow them to export the items mapped to a
													chosen standard instead of in its original form.
													
													The exporter will also be able to inform what mappers it supports. See `set_accepted_mapping_methods`
													and `accept_no_mapping` below.
	 											
	 												Note that this will only work when exporting items from one single collection.
	 
	 	   @type bool		 $manual_collection		Wether Tainacan will let the user choose the source collection.
	 	 											If set to true, Tainacan give the user a select box from where he/she will Choose
													one (and only one) Collection to export items from.
	 											
	 												Otherwise, the child exporter class must choose the collections somehow
													(it could be in its options form) and add them to the collections property using `add_collection()`
```

Note that depending on the value of `manual_mapping` and `manual_collection` you will have to implement some methods in your exporter class.

## Initializing a new exporter

When the user starts a new export process, he/she first choose which export to use.

Once the Exporter is chosen, the first thing that happens is the creation of a new instance of the chosen Exporter. This fires the `__construct()` method.


## Choose a collection (if `manual_collection` is true)

After choosing the exporter, user will be given the choice to choose the source collection.

If called from inside a collection, this step is skipped and the current collection is set as source.

## Set options

Now its time to set the exporter options. Each exporter may have its own set of options, that will be used during the export process. It could be anything, from the delimiter character in a CSV exporter, to an API key for a exporter that sends data to an external API.

exporter classes should declare the default values for its options in the `__construct()` method, by calling `set_default_options()`.

```
namespace Tainacan\exporter;

class Myexporter extends Exporter {
	function __construct() {
		parent::construct();
		$this->set_default_options(['
			'foo' => 'bar'
		']);
	}
}
```

The exporter classes must also implement the `options_form` method, in which it will echo the form that the user will interact to set the options.

```
	function options_form() {
		$form = '<div class="field">';
        $form .= '<label class="label">' . __('My exporter Option 1', 'tainacan') . '</label>';
		$form .= '<div class="control">';
		$form .= '<input type="text" class="input" name="my_exporter_option_1" value="' . $this->get_option('my_exporter_option_1') . '" />';
		$form .= '</div>';
		$form .= '</div>';

		return $form;
	}
```
## Accepting mappers 

If you set `manual_mapping` as true and want to give users the ability to export a mapped version of their collections, you can also 
define which mappers you accept. 

```
public function __construct($attributes = array()) {
	parent::__construct($attributes);
	$this->set_accepted_mapping_methods('any'); // set all method to mapping
	$this->accept_no_mapping = true;
}

```

`$this->set_accepted_mapping_methods()` lets you do that. If you set it to "any", all mappers will be available. If you set it to "list", you can then pass a second argument with the list of mappers you want to be available for the users.

```
public function __construct($attributes = array()) {
	parent::__construct($attributes);
	$this->set_accepted_mapping_methods('list', [ "dublin-core" ]); // set specific list of methods to mapping
}

```

Finally, `$this->accept_no_mapping = true;` informs that you also allow users to export items in their original form, without any mapping. In other words, to choose a Mapper is not mandatory if this is set to true.

## exporter steps

An exporter may have several steps, that will handle different parts of the process. Each step will be handled by a different callback in the exporter class.

First, lets have a look at a simple CSV exporter, that only have one step, in which it exports the items from the source collection into a CSV file. After that we will have a look on how to create custom steps.

### Simple exporter - One step that exports items

By default, exporters must implement 3 methods: `output_header`, `output_footer` and `process_item`.

`output_header` and `output_footer` are invoked at the beginning and end of each collection being exported. It can be useful, for example, to write the header and footer of a csv or xml file.

Inside this methods you may do whatever you want. It could be a POST to an API or to write a new file (see handling files below).

The `process_item` method will be invoked for every item being exported. Tainacan will automatically loop through the collections in the queue, fetch and prepare the items, and then send them, one at a time, to this method. (see adding collections below).

This method gets two parameters, the `$item` object (instance of \Tainacan\Entities\Item), and its `$metadata`, with an array of '\Tainacan\Entities\Item_Metadata_Entity' objects.

Note that, in this method, you dont need to bother about mapping. If a mapper was chosen by the user, you will receive this array of metadata already mapped, which means they will be only the ones defined in the mapper. (Note that, if a collection fail to map one of its metadata to a metadata expected by the chosen mapper, they will be empy elements in this array).

Now you can loop through the metadata and access any propery you want from the item to do whatever you like. In the case of the CSV exporter, it will add one line to the CSV file.

If you need to access the Mapper object, with everything about the the chosen mapper standard, you can do so by calling `$this->get_current_mapper()`. If a object is not returned, it means no mapper was selected by the user.

#### Adding collections

If your exporter does not use the `manual_collection` option, you might want to programatically add collections to be exported to the queue.

To add or remove a collection from the queue, use the `add_collection()` and `remove_collection()` methods passing the collection definition.

The collection definition is an array with their IDs and the total number of items to be exported.


Example of the structure of this propery for one collection:

```
[
	'id' => 12,
	'total_items' => 1234,
]
```

#### Handling files 

Your export may generate one or more files, that will be downloaded by the user as a package at the end.

To create and write contents to a file, use the `append_to_file()` method.

It takes 2 arguments: `$key` and `$data`. `$key` is the file identifier and will serve as the filename, prepended with the ID of the process. If a file with this key does not exist yet, it will be created. `$data` is the content to be appended to the file.

Dont forget to add a link to the generated file at the output at the end of the process. This will be the only way users will have to get them. See "Final output" below).

TODO: If more than one file was created, Tainacan will create a zip of them at the end. Yet to be implemented.

#### Using transients

Since exporters are handled as background processes, they will run accross multiple requests. For that reason, you can not simply add properties to your class and trust their values will be kept during all the time the process is running.

If you want a value to persist so you can use it accross all methods of your exporter, at any time, you should use `transients` to store them in the database.

This is as simple as calling `set_transient()` and `get_transient()`. See the example below:


```

function callback_for_step_1() {
	
	
	$this->add_transient('time_step_1', time());
	
}

// ...

function callback_for_step_5() {
	
	$time_step_1 = get_transient('time_step_1');
	
}

```


#### Handling user feedback

There are two information Tainacan exporters give to the user about the status of the process while it is running in feedback. the `progress label` and the `progress value`. 

The `progress label` is a string that can be anything that tells the user what is going on. By default, it is the Step Name, but you can inform a specific `progress_label` attribute when registering the steps.

The `progress value` is a number between 0 and 100 that indicates the progress of the current step or the whole exporter, Thats up to you. By default, it calculates it automatically using the `total` attribute registered with the steps, against the `$this->get_in_step_count()` value. In the case of the default Process Items callback, it calculates based on the number of items found in each collection.

See the `finish_processing` dummy callback in the Test Importer. You will notice that when we registered the step, we informed a `total` attribute to this step with the value of 5. This will tell Tainacan that the total number iterations this step need to complete is 5 and allow it to calculate the progress.

If it is no possible to know `total` of a step beforehand, you can set it at any time, even inside the step callback itself, using the `set_current_step_total($value)` or `set_step_total($step, $value)` methods.


##### Final output 

When the process finishes, Background processes define an "output", which is a final report to the user of what happened.

This could be simply a message saying "All good", or could be a report with the names and links to the collections created. HTML is welcome.

Remember that for a more detailed and technical report, you should use Logs (see Logs below). This output is meant as a short but descriptive user friendly message.

In order to achieve this, exporters must implement a method called `get_output()` that returns a string.

This method will be called only once, when the exporter ends, so you might need to save information using `transients` during the process and use them in `get_output()` to compose your message.

To get a list of the generated files to display to users, use `$this->get_output_files()`.

#### Logs

There are two useful methods to write information to the logs: `add_log()` and `add_error_log()`. These are written into a log file related to the exporter background process and a link to it will be presented to the user.



## Run exporter

Finally, everything is ready. The exporter runs.

This will trigger a Background Process (documentation needed) and the exporter will run through as many background requests as needed.