Nasqueron Datasources: Difference between revisions

From Nasqueron Agora
mNo edit summary
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
'''Nasqueron Datasources'' is a collection of tools and libraries, mostly written in Rust, to fetch and enrich data.
'''Nasqueron Datasources''' is a collection of tools and libraries, mostly written in Rust, to fetch and enrich data.


== Datasources ==
== Datasources ==
Line 9: Line 9:
During that operation, it can generate [https://www.wikidata.org/wiki/User:Dereckson/FANTOIR_maintenance_report a maintenance report].
During that operation, it can generate [https://www.wikidata.org/wiki/User:Dereckson/FANTOIR_maintenance_report a maintenance report].


<source>
See [[fantoir-datasource]] for how to use instructions.
$ fantoir-datasource help
 
Import FANTOIR database into PostgreSQL
 
Usage: fantoir-datasource <COMMAND>
 
Commands:
  import    Import from FANTOIR file generated by the DGFIP
  promote  Promote an imported FANTOIR table as the current FANTOIR table to use
  wikidata  Query Wikidata SPARQL end-point to enrich FANTOIR information
  query    Query the imported FANTOIR table
  help      Print this message or the help of the given subcommand(s)
</source>


=== Helper components ===
=== Helper components ===

Latest revision as of 01:42, 18 January 2023

Nasqueron Datasources is a collection of tools and libraries, mostly written in Rust, to fetch and enrich data.

Datasources

Datasources components

FANTOIR (fantoir-datasource)

fantoir-datasource is a CLI tool to import the FANTOIR database into PostgreSQL. It's built with pipeline as a code in mind, so we can run it when a new FANTOIR version is released.

It also can enrich data from Wikidata, to qualify ways and offer rich typography labels. During that operation, it can generate a maintenance report.

See fantoir-datasource for how to use instructions.

Helper components

The datasources repository contains the following libraries:

  • The opendatasoft-explore-api crate allows to query the Opendatasoft Explore API from Rust code. This API software is for example used for data.economie.gouv.fr for open data.

A SPARQL client can also be extracted from fantoir-datasource to be published as a standalone library (and so be used in outside projects).

Dependencies

The following relations exist in our components:

  • fantoir-datasource uses opendatasoft-explore-api to check if a new version is available and get the URL

Contribute

The Nasqueron Databases project on DevCentral is used for any issue, code idea, bug or task.

Repository to clone: datasources

Contact: Dereckson is willing to help you if you need to discuss this project.

Licences

Source code is released under BSD-2-Clause license.

The datasets we provide tooling for and consume are and must be licensed under free culture licences, tailored for sui generis database rights:

  • LO (e.g. FANTOIR)
  • ODbL (e.g. OpenStreetMap)

The datasets we produce are licensed under:

  • when a copyleft applies, for example when a dataset is ODbL, ODbL
  • when no copyleft applies, because all sources allow redistribution under any license: CC-BY 4.0