¶ Creating the first application
Pyplan is an integrated development environment for applications, designed for users without programming knowledge. This is why the users build the calculation logic and data processing of the applications by dragging different types of nodes in an influence diagram.
Clicking "Create an application" opens a dialog box that lets you choose to create an application from scratch or from an example.
By choosing to create a data analysis application and then selecting the code visualization, you will reach a window like the following:
¶ Elements of the coding window
Once you have clicked on the code icon in the main menu, you will access the influence diagram:
The elements that make up this
section of the platform are:
- Influence diagram: Graphical
representation of data processing
up to its transformation into output.
- Node display types: Result |
Code+Result | Code+Preview.
- Help bot: Robot assistant that
interprets natural language to
answer questions about the use
of Pyplan.
- Navigation Shortcuts: Link to Pyplan
Home and application Home,
accessible by clicking on
the Pyplan logo and the application
title, respectively.
- Properties of an application: Drop-down menus when you click on the three dots to the right of the app title.
¶ Influence Diagram
It is the graphical representation of the calculation process. Each node represents a calculation step in the data transformation process. A node is selected by clicking on it. By double-clicking on a node, it is calculated. Its result is displayed in full screen according to your default display settings. Links (arrows) between nodes are automatically generated when invoking a node within the definition of another node.
¶ Features of a Node
A node is the smallest unit of construction of the calculation process. To add a node, drag it onto the diagram.
Each node has the following properties:
¶ Node title
It is the text displayed inside the node in the diagram
¶ Node Id
It is the identifier of the node, that is, the way in which a node is invoked in the definition of another node. The node’s Id is automatically generated from the title, but can be changed by the user.
¶ Unit
If a node contains a single measure, it is possible to indicate its unit. This will be shown in parentheses next to the title when the node’s results are displayed.
It is worth noting that units are only included for visualization purposes, they are not part of the calculation of results.
¶ Evaluate on start
If enabled, the node will be automatically evaluated when the application opens.
¶ Release memory
When enabled, the node’s result is not stored in RAM after evaluation. This helps freeing up memory for intermediate nodes that have a single output. If the node’s result is needed by another node, it will be recalculated.
¶ Definition
It is the transformation operation executed when evaluating a node. Its code is written in Python language. It ends with the definition of its result “result=”
¶ Preview
It is the technical description of the output of the calculation process.
¶ Result
It is the result of executing the code of the definition. When this result is an object interpreted by Pyplan (a Pandas dataframe, a Numpy matrix, or a matrix with designated dimensions of Xarray), it can be visualized graphically or in table form.
¶ Documentation
It is the text that conceptually describes each calculation step and is displayed at the bottom of the result view.
¶ Inputs
List of nodes used in the definition of a node.
¶ Outputs
List of nodes that use the result of the selected node.
You can access all of these properties by right-clicking on the node and then clicking on properties.
¶ Node types
There are different types of nodes for different purposes. Their functionality has been designed to facilitate the definition of the parameters necessary for its execution.
¶ Variable
Variable type nodes are the most commonly used, as they have the function of containing a generic calculation process. They are blue in color initially. If it does not have outputs, this color changes to gray and it is considered a report. It changes to red if the outputs of the node are outside the module to which it belongs.
¶ Data reading
Node that when dragged displays an assistant to connect to different data sources:
¶ Input data
Node that allows to create a manual data entry process.
Once the type of data input is chosen, an assistant is displayed for its configuration.
¶ Form
¶ Cube
An InputCube type node is a node that allows entering data in a multidimensional manner as if it were a pivot table. The table values are saved in a configurable database within the same wizard.
The initial configuration looks like the following:
- Data type: It indicates the type that the node will accept as values.
- Default value type: Can be “Scalar” or “App node”.
- Default value (optional): Default value to be input for those concepts to which no value is assigned. In case Default value type is “Scalar”, this field accepts atomic values such as 0 or ‘empty’ (it must comply with the data type defined before). In case Default value type is “App node”, it indicates the identifier of the node that contains the default value. The node result can be a scalar or an xarray DataArray. In the latter case, the field is a node finder by identifier or title.
- Available indexes / Selected indexes: This is where the Index type nodes are chosen as the dimensions of the table.
The extra settings you can configure are the following:
- Table name: Name of the table that will be saved in the database. It defaults to the node identifier.
- Database engine: Can be “sqlite”, “postgresql” or “custom”.
- Database parameters: Depending on which Database engine it was chosen, you will have to configure different parameters in order to successfully connect to that database.
- Remove non-existing combinations: If checked, removes any combination of dimensions that is not present in the actual set of combinations from given dimensions. This can happen if the values in one of the chosen dimensions changes (e.g., “Categories” dimension changes from ["Category A", “Category B”] to ["Category B", “Category C”; in this case, all values saved as “Category A” will be deleted if this option is checked).
- Allow dimension changes: If unchecked, if a new dimension is added or removed after the table was created, it raises an error.
Important considerations:
- To access the xarray DataArray object underneath this node, you must invoke the “value” property of it. e.g., “result = sales_data.value” .
¶ Report
The Report type node is a node that allows creating a report from a list of concepts to which a value or the result of a node is assigned for each concept.
The "Items from index" checkbox indicates whether the list of concepts in the report comes from an Index node (whose result is a pandas Index) or not. In the latter case, the list of concepts is generated manually.
If "Items from index" is unchecked (as shown in the image), it is necessary to complete:
- Dimension name: The name of the dimension that will be inserted into the report, and its values will be the list of concepts.
- Default value (optional): Default value to be input for those concepts to which no value or node is assigned.
For this variation, it is possible to add new concepts through the "Add new item" button. Additionally, it is possible to change the order of already created concepts, delete, and modify them.
If "Items from index" is checked, the dialog looks like the following:
In this case, it is necessary to complete:
- Index node: Identifier of a node whose result is a pandas Index and where the node is of type Index. It is a node finder for Index type nodes by identifier or title.
- Default value (optional): Default value to be input for those concepts to which no value or node is assigned.
For this variant, once the "Index node" field is completed, it is possible to automatically generate all the concepts of the dimension by pressing the "Add items from index" button.
Once the concepts are added and filled out, it will look similar to the following image:
Each concept has the following properties:
- Item name: Name of the concept.
- Source: Can be “Input” or “Node”.
- Value: In case "Source" is "Input," this property receives a scalar indicating the value that the report will have for the given Item name. If "Source" is "Node," it indicates the identifier of the node that contains the values to assign for the given item name. In the latter case, the field is a node finder by identifier or title.
- As negative: If checked, the result of the concept is multiplied by -1 when generating the report. This field is only enabled if "Source" is "Node". It may be useful in situations where the the node that holds the result of your concept is expressed in positive values but you want to show them as negative in the report. For example, when generating a Profit & Loss financial statement, you may want to exhibit the “Cost of Goods Sold” (COGS) as negative values compared to the “Sales” concept, which should be positive.
Finally, the "Output type" selector allows choosing the output format that the report will have, with the following options:
- xarray DataArray: The final result will be an xarray DataArray where the array will have all the dimensions existing in each of the xarray DataArray type nodes assigned to each concept.
- pandas DataFrame: The final result will be a dataframe that concatenates all the dataframes assigned in each of the pandas DataFrame type nodes that make up the report.
Important considerations:
- If the "Output type" is "xarray DataArray," the report only supports nodes whose result is xarray DataArray or scalars. On the other hand, if the "Output type" is "pandas DataFrame," the report only supports nodes whose result is pandas DataFrame, pandas Series, or scalars.
- When “Items from index” is checked, the concepts list in the report is binded to the concepts list in the assigned node: if a new concept is added to the original list in the linked node, that concept will be automatically created in the report, assigning the default value for its value until a new value is allocated to this concept. The report will always show all the concepts that are available in the linked node.
- To access the xarray DataArray or pandas DataFrame object underneath this node, you must invoke the “value” property of it. e.g., “result = financial_report.value”.
¶ Button
The button is a special node that allows you to execute Python code each time the user presses it. In this type of nodes, the result does not persist in memory.
¶ Index
The index node is the way Pyplan stores dimensions that are used throughout the calculation process. Its definition leads to the determination of a list of elements that constitute the dimension. They have a particular functioning when included in interfaces.
¶ Text
The text box is a construction element of the diagram for the sole purpose of facilitating the interpretation of the influence diagram.
¶ Module
It is a special type of node that allows you to contain nodes inside it. It serves to hierarchically organize a complex calculation. A module is entered by double-clicking on it. Modules can be nested indefinitely.
¶ Alias
Aliases are a type of node that are used to make influence diagrams more explanatory. They are a mirror of the original node. That is, not only does the result they show match the original, but modifying the definition of the alias also modifies the definition of the original node. The Alias is created by pressing Ctrl+M once a node is selected.
¶ Operations in the Influence Diagram
¶
Navigation
The influence diagram is a hierarchical representation of the calculation logic. Scrolling within the diagram is done by double-clicking on the module you want to enter or by clicking on the path generated from the navigation to move up to the higher hierarchies.
¶ Inspection of node properties
When selecting a node, right-clicking on it displays a node properties window.
¶ Drag nodes to the diagram
Clicking on the blue button at the upper left corner displays the different types of nodes to drag onto the diagram.
This is the most commonly used way to create new calculation steps in the data processing flow of the application.
It is also possible to copy <Ctrl+C>, cut <Ctrl+X> and paste <Ctrl+V> a selected node or group of nodes. To select multiple nodes, click on each node while keeping the <Ctrl> key pressed. You can also generate a selection rectangle by holding down the <Shift> key and clicking on the diagram and then scrolling to delimit the rectangle.
¶ Results display
By selecting a node, we can know if it is calculated or not, as indicated in the result window:
A node is calculated by double-clicking on it or by pressing the Play icon that appears in the result window or by pressing <Ctrl+Enter> once the node is selected.
There are three types of visualizations of a node that can be chosen in the upper right part of the diagram window.
- The first corresponds to the display of results together with the documentation.
- The second one allows you to see the definition (code) of the node together with the result of the node.
- The third one allows to view the code of the node definition together with an inspection of the technical characteristics of the resulting object.