File Ingestion

file ingestion lets you define specifications for incoming files so that cloud management can process them each ingestion specification identifies a source folder and the file format to expect, such as csv, avro, or json access file ingestion in cloud management > file ingestion file ingestion list the file ingestion list shows all defined ingestion specifications column description name name of the ingestion specification folder name folder pattern that identifies where incoming files are located source file format of the incoming files, such as csv, avro, or json pack configuration pack the specification belongs to enabled whether the ingestion specification is enabled actions options to edit or delete the specification to include disabled specifications in the list, select show disabled create an ingestion specification go to cloud management > file ingestion click add new ingestion specification in the properties section, enter an ingestion name to enable the specification immediately, select enabled in the folder name field, enter the relative path in the gcs folder where the incoming files are located in the source type field, select the file format of the incoming files csv , avro , avro raw , or json complete the attributes section the available fields depend on the source type selected click create csv attributes when csv is selected as the source type, the following attributes are available in the attributes section field description separator character used to separate columns in the csv file skip rows number of rows to skip before reading data the default value is 0 include file attributes whether to include file level attributes in the ingested data use header row as field names whether to use the values in the header row as field names