Data can be imported from one or more sources. Data is imported using connectors, which can read data from places such as files or databases.
Import data from one source.
read:
- file:
name: file.xlsx
sheet_name: my sheet
Combine data from multiple sources, stacked vertically, equivalent to a union in SQL.
Where headers differ, columns will be matched by name.
read:
- union:
sources:
- file:
name: file1.xlsx
- file:
name: file2.xlsx
|
+ |
|
→ |
|
Union is the default aggregation for multiple reads.
Combine data from multiple sources, stacked horizontally.
read:
- concatenate:
sources:
- file:
name: file1.xlsx
- file:
name: file2.xlsx
|
+ |
|
→ |
|
Joins can be done using data from any connector, or even custom functions.
Want to join a csv to data from a database? Go for it!
Import data from two sources and merge the data, equivalent to a join in SQL.
This can be used recursively to join more than two tables.
read:
- join:
how: inner
left_on: column
right_on: column
sources:
- file:
name: file1.xlsx
- file:
name: file2.xlsx