Importer Service
TopGraphRAGImporterService handles file imports for GraphRAG creation
HTTP Bindings
| Endpoint | Method | RPC Method | Body |
/v1/import |
POST | ImportFile | * |
/v1/import-multiple |
POST | ImportMultipleFiles | * |
/v1/recluster |
POST | ReclusterPartition | * |
/v1/jobs/{job_id} |
GET | GetJobStatus | |
/v1/jobs |
GET | ListJobs | |
/v1/health |
GET | HealthCheck |
Methods
| Method Name | Request Type | Response Type |
| ImportFile | ImportFileRequest | ImportFileResponse |
|
ImportFile processes a file to create a GraphRAG |
||
| ImportMultipleFiles | ImportMultipleFilesRequest | ImportMultipleFilesResponse |
|
ImportMultipleFiles processes multiple files to create a single GraphRAG Returns immediately after files are validated and saved, with a job_id for tracking |
||
| ReclusterPartition | ReclusterRequest | ReclusterResponse |
|
ReclusterPartition re-runs Leiden clustering + community reports for one partition (AIS-1444). Returns immediately with a job_id; poll GetJobStatus. |
||
| GetJobStatus | GetJobStatusRequest | GetJobStatusResponse |
|
GetJobStatus retrieves the current status and history of an import job |
||
| ListJobs | ListJobsRequest | ListJobsResponse |
|
ListJobs retrieves all jobs with their current status |
||
| HealthCheck | HealthCheckRequest | HealthCheckResponse |
|
HealthCheck provides basic service health status |
||
FileInput
FileInput represents a single file with metadata for multi-file processing Note: image flags (enable_semantic_units/process_images/enable_semantic_unit_embeddings) are configured at ImportMultipleFilesRequest level and apply to the full batch.
| Field | Type | Label | Description |
| name | string | Original filename |
|
| content | bytes | File content in bytes |
|
| citable_url | string | URL to be cited in inline citations |
GetJobStatusRequest
GetJobStatusRequest requests the status of a specific import job
| Field | Type | Label | Description |
| job_id | string | The job identifier to query |
GetJobStatusResponse
GetJobStatusResponse returns the complete status of an import job
| Field | Type | Label | Description |
| success | bool | Whether the job was found |
|
| error_message | string | optional | Error message if job not found |
| job | JobStatus | optional | The job status details (if found) |
HealthCheckRequest
Health check messages
HealthCheckResponse
| Field | Type | Label | Description |
| success | bool |
|
|
| message | string | optional |
|
ImportFileRequest
ImportFileRequest represents the input file for processing
| Field | Type | Label | Description |
| file_content | bytes | Direct file content |
|
| file_url | string | URL to download the file from |
|
| file_name | string | Original filename |
|
| store_in_s3 | bool | Whether to store in S3 |
|
| batch_size | int32 | Batch size for importing documents, entities, and relationships |
|
| enable_chunk_embeddings | bool | Whether to enable chunk embeddings |
|
| enable_edge_embeddings | bool | Whether to enable edge embeddings |
|
| chunk_token_size | int32 | Maximum tokens per chunk |
|
| chunk_overlap_token_size | int32 | Overlap tokens between chunks |
|
| entity_types | string | repeated | Entity types to extract |
| relationship_types | string | repeated | Relationship types to extract |
| partition_id | string | Partition id |
|
| enable_semantic_units | bool | Enable semantic units (web URLs; extract embedded PDF images + FileManager upload when true) |
|
| process_images | bool | Generate multimodal descriptions (requires enable_semantic_units) |
|
| chunk_min_token_size | int32 | Minimum tokens per chunk (optional) |
|
| chunk_custom_separators | string | repeated | Custom separators for chunking (optional) |
| preserve_chunk_separator | bool | Whether to preserve separator in chunks (optional) |
|
| smart_graph_attribute | string | Smart Graph Attribute |
|
| shard_count | int32 | Must be 1 for new SmartGraph or sharded enterprise graph (IVF index per-shard; empty shards -> ERR 9). Other values rejected. |
|
| is_disjoint | bool | Whether the graphs must be disjoint |
|
| satellite_collections | string | repeated | An array of collection names that is used to create Satellite Collections |
| enable_strict_types | bool | Enable strict filtering of entities and relationships based on provided types |
|
| entity_extract_max_gleaning | int32 | default is 1 |
|
| vector_index_n_lists | int32 | optional | Number of lists for approximate search. If not set, computed as 8 * sqrt(collection_size) |
| vector_index_metric | string | optional | Distance metric for vector index (default: "cosine") |
| vector_index_use_hnsw | bool | Whether to use HNSW index (defaults to False) |
|
| enable_community_embeddings | bool | optional | Whether to enable community embeddings (default true when unset) |
| ignore_chunk_token_size | bool | If true, chunk by separators only (ignores token size limits) |
|
| custom_prompts | ImportFileRequest.CustomPromptsEntry | repeated | Dictionary of custom prompts (key: prompt name, value: prompt text). If non-empty, custom prompts will be used instead of defaults. |
| rag_mode | string | "vector_rag" or "full_graphrag" (empty defaults to full_graphrag) |
|
| file_id | string | PATH B: RAG file ID from File Manager. When set, source (file_content/file_url) is ignored and the file is fetched by ID. |
|
| enable_semantic_unit_embeddings | bool | Whether to enable semantic unit embeddings |
ImportFileRequest.CustomPromptsEntry
| Field | Type | Label | Description |
| key | string |
|
|
| value | string |
|
ImportFileResponse
| Field | Type | Label | Description |
| success | bool |
|
|
| message | string | optional |
|
ImportMultipleFilesRequest
ImportMultipleFilesRequest represents multiple files for processing into a single GraphRAG
| Field | Type | Label | Description |
| files | FileInput | repeated | List of files to process |
| store_in_s3 | bool | Whether to store in S3 |
|
| batch_size | int32 | Batch size for importing documents, entities, and relationships |
|
| enable_chunk_embeddings | bool | Whether to enable chunk embeddings |
|
| enable_edge_embeddings | bool | Whether to enable edge embeddings |
|
| chunk_token_size | int32 | Maximum tokens per chunk |
|
| chunk_overlap_token_size | int32 | Overlap tokens between chunks |
|
| entity_types | string | repeated | Entity types to extract |
| relationship_types | string | repeated | Relationship types to extract |
| partition_id | string | Partition id |
|
| enable_semantic_units | bool | Enable semantic units (web URLs; extract embedded PDF images + FileManager upload when true) |
|
| process_images | bool | Generate multimodal descriptions (requires enable_semantic_units) |
|
| chunk_min_token_size | int32 | Minimum tokens per chunk (optional) |
|
| chunk_custom_separators | string | repeated | Custom separators for chunking (optional) |
| preserve_chunk_separator | bool | Whether to preserve separator in chunks (optional) |
|
| smart_graph_attribute | string | Smart Graph Attribute |
|
| shard_count | int32 | Must be 1 for new SmartGraph or sharded enterprise graph (IVF index per-shard; empty shards -> ERR 9). Other values rejected. |
|
| is_disjoint | bool | Whether the graphs must be disjoint |
|
| satellite_collections | string | repeated | An array of collection names that is used to create Satellite Collections |
| enable_strict_types | bool | Enable strict filtering of entities and relationships based on provided types |
|
| entity_extract_max_gleaning | int32 | Maximum number of extraction iterations to refine entity extraction (default is 1) |
|
| vector_index_n_lists | int32 | optional | Number of lists for approximate search. If not set, computed as 8 * sqrt(collection_size) |
| vector_index_metric | string | optional | Distance metric for vector index (default: "cosine") |
| vector_index_use_hnsw | bool | Whether to use HNSW index (defaults to False) |
|
| enable_community_embeddings | bool | optional | Whether to enable community embeddings (default true when unset) |
| ignore_chunk_token_size | bool | If true, chunk by separators only (ignores token size limits) |
|
| custom_prompts | ImportMultipleFilesRequest.CustomPromptsEntry | repeated | Dictionary of custom prompts (key: prompt name, value: prompt text). If non-empty, custom prompts will be used instead of defaults. |
| rag_mode | string | "vector_rag" or "full_graphrag" (empty defaults to full_graphrag) |
|
| file_ids | string | repeated | PATH B: RAG file IDs from File Manager. When non-empty, files are fetched by ID and files (field 1) are ignored. |
| enable_semantic_unit_embeddings | bool | Whether to enable semantic unit embeddings |
ImportMultipleFilesRequest.CustomPromptsEntry
| Field | Type | Label | Description |
| key | string |
|
|
| value | string |
|
ImportMultipleFilesResponse
ImportMultipleFilesResponse is returned after files are validated and saved
| Field | Type | Label | Description |
| success | bool | Whether files were processed successfully |
|
| job_id | string | Unique job identifier for tracking progress |
|
| message | string | optional | Success or error message |
| error_message | string | optional | Detailed error message if success is false |
| service_id | string | optional | Service/pod identifier for orchestrator job→pod mapping (same as GenAI metadata) |
ImportProgressResponse
ImportProgressResponse provides streaming updates during multi-file processing
| Field | Type | Label | Description |
| type | ImportProgressType | Type of progress update |
|
| message | string | Progress message |
|
| current_file_index | int32 | Current file being processed (0-based) |
|
| total_files | int32 | Total number of files |
|
| success | bool | Whether the operation succeeded |
|
| error_message | string | optional | Error message if success is false |
JobStatus
JobStatus represents the complete status of an import job
| Field | Type | Label | Description |
| job_id | string | Unique job identifier |
|
| created_at | string | ISO 8601 timestamp of job creation |
|
| files | string | repeated | List of file names being processed |
| files_count | int32 | Number of files in the job |
|
| current_status | StatusEntry | optional | Current status of the job |
| is_terminal | bool | Whether the job has finished (completed or failed) |
|
| status_history | StatusEntry | repeated | Chronological history of status updates (oldest first) |
JobSummary
JobSummary provides a brief overview of a job for listing
| Field | Type | Label | Description |
| job_id | string | Unique job identifier |
|
| created_at | string | ISO 8601 timestamp of job creation |
|
| files | string | repeated | List of file names being processed |
| files_count | int32 | Number of files in the job |
|
| current_status | string | Current status name |
|
| is_terminal | bool | Whether the job has finished |
ListJobsRequest
ListJobsRequest requests a list of all jobs
| Field | Type | Label | Description |
| limit | int32 | Maximum number of jobs to return (default: 50) |
ListJobsResponse
ListJobsResponse returns a list of all jobs
| Field | Type | Label | Description |
| success | bool | Whether the request succeeded |
|
| jobs | JobSummary | repeated | List of job summaries (most recent first) |
ReclusterRequest
ReclusterRequest asks the importer to recluster a single partition (AIS-1444).
| Field | Type | Label | Description |
| partition_id | string | The rag_partition_id to recluster |
ReclusterResponse
ReclusterResponse returns the background job id for the recluster (AIS-1444).
| Field | Type | Label | Description |
| success | bool | Whether the recluster was accepted |
|
| job_id | string | Job id for GET /v1/jobs/{job_id} |
|
| message | string | optional | Success or error message |
| service_id | string | optional | Service/pod identifier |
StatusEntry
StatusEntry represents a single status update in a job's history
| Field | Type | Label | Description |
| status | string | Status name (e.g., "GRAPH_BUILDER_STARTED") |
|
| timestamp | string | ISO 8601 timestamp of this status update |
|
| progress | float | Progress percentage (0.0 to 100.0) |
|
| message | string | Optional message with additional details |
ImportProgressType
ImportProgressType defines the types of progress updates
| Name | Number | Description |
| IMPORT_PROGRESS_TYPE_UNSPECIFIED | 0 | |
| IMPORT_PROGRESS_TYPE_FILES_RECEIVED | 1 | All files received and validated |
| IMPORT_PROGRESS_TYPE_FILE_PROCESSING | 2 | Currently processing a specific file |
| IMPORT_PROGRESS_TYPE_GRAPH_BUILDING | 3 | Building the knowledge graph |
| IMPORT_PROGRESS_TYPE_DATABASE_IMPORT | 4 | Importing to ArangoDB |
| IMPORT_PROGRESS_TYPE_COMPLETED | 5 | All processing completed successfully |
| IMPORT_PROGRESS_TYPE_ERROR | 6 | Error occurred during processing |
Scalar Value Types
| .proto Type | Notes | C++ | Java | Python | Go | C# | PHP | Ruby |
| double | double | double | float | float64 | double | float | Float | |
| float | float | float | float | float32 | float | float | Float | |
| int32 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| int64 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| uint32 | Uses variable-length encoding. | uint32 | int | int/long | uint32 | uint | integer | Bignum or Fixnum (as required) |
| uint64 | Uses variable-length encoding. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum or Fixnum (as required) |
| sint32 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sint64 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | int | uint32 | uint | integer | Bignum or Fixnum (as required) |
| fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum |
| sfixed32 | Always four bytes. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sfixed64 | Always eight bytes. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| bool | bool | boolean | boolean | bool | bool | boolean | TrueClass/FalseClass | |
| string | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String | str/unicode | string | string | string | String (UTF-8) |
| bytes | May contain any arbitrary sequence of bytes. | string | ByteString | str | []byte | ByteString | string | String (ASCII-8BIT) |