airbyte.DestinationSnowflakeCortex

Coming soon!

Coming soon!

Coming soon!

Coming soon!

name string

The unique name of the resource.

args DestinationSnowflakeCortexArgs

The arguments to resource properties.

opts CustomResourceOptions

Bag of options to control resource's behavior.

resource_name str

The unique name of the resource.

args DestinationSnowflakeCortexArgs

The arguments to resource properties.

opts ResourceOptions

Bag of options to control resource's behavior.

ctx Context

Context object for the current deployment.

name string

The unique name of the resource.

args DestinationSnowflakeCortexArgs

The arguments to resource properties.

opts ResourceOption

Bag of options to control resource's behavior.

name string

The unique name of the resource.

args DestinationSnowflakeCortexArgs

The arguments to resource properties.

opts CustomResourceOptions

Bag of options to control resource's behavior.

name String

The unique name of the resource.

args DestinationSnowflakeCortexArgs

The arguments to resource properties.

options CustomResourceOptions

Bag of options to control resource's behavior.

Configuration DestinationSnowflakeCortexConfiguration

The configuration model for the Vector DB based destinations. This model is used to generate the UI for the destination configuration, as well as to provide type safety for the configuration passed to the destination.

WorkspaceId string

DefinitionId string

The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.

Name string

Name of the destination e.g. dev-mysql-instance.

Configuration DestinationSnowflakeCortexConfigurationArgs

The configuration model for the Vector DB based destinations. This model is used to generate the UI for the destination configuration, as well as to provide type safety for the configuration passed to the destination.

WorkspaceId string

DefinitionId string

The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.

Name string

Name of the destination e.g. dev-mysql-instance.

configuration DestinationSnowflakeCortexConfiguration

The configuration model for the Vector DB based destinations. This model is used to generate the UI for the destination configuration, as well as to provide type safety for the configuration passed to the destination.

workspaceId String

definitionId String

The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.

name String

Name of the destination e.g. dev-mysql-instance.

configuration DestinationSnowflakeCortexConfiguration

The configuration model for the Vector DB based destinations. This model is used to generate the UI for the destination configuration, as well as to provide type safety for the configuration passed to the destination.

workspaceId string

definitionId string

The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.

name string

Name of the destination e.g. dev-mysql-instance.

configuration DestinationSnowflakeCortexConfigurationArgs

The configuration model for the Vector DB based destinations. This model is used to generate the UI for the destination configuration, as well as to provide type safety for the configuration passed to the destination.

workspace_id str

definition_id str

The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.

name str

Name of the destination e.g. dev-mysql-instance.

configuration Property Map

The configuration model for the Vector DB based destinations. This model is used to generate the UI for the destination configuration, as well as to provide type safety for the configuration passed to the destination.

workspaceId String

definitionId String

The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.

name String

Name of the destination e.g. dev-mysql-instance.

CreatedAt double

DestinationId string

DestinationType string

Id string

The provider-assigned unique ID for this managed resource.

CreatedAt float64

DestinationId string

DestinationType string

Id string

The provider-assigned unique ID for this managed resource.

createdAt Double

destinationId String

destinationType String

id String

The provider-assigned unique ID for this managed resource.

createdAt number

destinationId string

destinationType string

id string

The provider-assigned unique ID for this managed resource.

created_at float

destination_id str

destination_type str

id str

The provider-assigned unique ID for this managed resource.

createdAt Number

destinationId String

destinationType String

id String

The provider-assigned unique ID for this managed resource.

name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

state

Any extra arguments used during the lookup.

opts

A bag of options that control this resource's behavior.

resource_name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

state

Any extra arguments used during the lookup.

opts

A bag of options that control this resource's behavior.

name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

state

Any extra arguments used during the lookup.

opts

A bag of options that control this resource's behavior.

name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

state

Any extra arguments used during the lookup.

opts

A bag of options that control this resource's behavior.

Embedding DestinationSnowflakeCortexConfigurationEmbedding

Embedding configuration

Indexing DestinationSnowflakeCortexConfigurationIndexing

Snowflake can be used to store vector data and retrieve embeddings.

Processing DestinationSnowflakeCortexConfigurationProcessing

OmitRawText bool

Do not store the text that gets embedded along with the vector and the metadata in the destination. If set to true, only the vector and the metadata will be stored - in this case raw text for LLM use cases needs to be retrieved from another source. Default: false

Embedding DestinationSnowflakeCortexConfigurationEmbedding

Embedding configuration

Indexing DestinationSnowflakeCortexConfigurationIndexing

Snowflake can be used to store vector data and retrieve embeddings.

Processing DestinationSnowflakeCortexConfigurationProcessing

OmitRawText bool

Do not store the text that gets embedded along with the vector and the metadata in the destination. If set to true, only the vector and the metadata will be stored - in this case raw text for LLM use cases needs to be retrieved from another source. Default: false

embedding DestinationSnowflakeCortexConfigurationEmbedding

Embedding configuration

indexing DestinationSnowflakeCortexConfigurationIndexing

Snowflake can be used to store vector data and retrieve embeddings.

processing DestinationSnowflakeCortexConfigurationProcessing

omitRawText Boolean

Do not store the text that gets embedded along with the vector and the metadata in the destination. If set to true, only the vector and the metadata will be stored - in this case raw text for LLM use cases needs to be retrieved from another source. Default: false

embedding DestinationSnowflakeCortexConfigurationEmbedding

Embedding configuration

indexing DestinationSnowflakeCortexConfigurationIndexing

Snowflake can be used to store vector data and retrieve embeddings.

processing DestinationSnowflakeCortexConfigurationProcessing

omitRawText boolean

Do not store the text that gets embedded along with the vector and the metadata in the destination. If set to true, only the vector and the metadata will be stored - in this case raw text for LLM use cases needs to be retrieved from another source. Default: false

embedding DestinationSnowflakeCortexConfigurationEmbedding

Embedding configuration

indexing DestinationSnowflakeCortexConfigurationIndexing

Snowflake can be used to store vector data and retrieve embeddings.

processing DestinationSnowflakeCortexConfigurationProcessing

omit_raw_text bool

Do not store the text that gets embedded along with the vector and the metadata in the destination. If set to true, only the vector and the metadata will be stored - in this case raw text for LLM use cases needs to be retrieved from another source. Default: false

embedding Property Map

Embedding configuration

indexing Property Map

Snowflake can be used to store vector data and retrieve embeddings.

processing Property Map

omitRawText Boolean

Do not store the text that gets embedded along with the vector and the metadata in the destination. If set to true, only the vector and the metadata will be stored - in this case raw text for LLM use cases needs to be retrieved from another source. Default: false

AzureOpenAi DestinationSnowflakeCortexConfigurationEmbeddingAzureOpenAi

Use the Azure-hosted OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

Cohere DestinationSnowflakeCortexConfigurationEmbeddingCohere

Use the Cohere API to embed text.

Fake DestinationSnowflakeCortexConfigurationEmbeddingFake

Use a fake embedding made out of random vectors with 1536 embedding dimensions. This is useful for testing the data pipeline without incurring any costs.

OpenAi DestinationSnowflakeCortexConfigurationEmbeddingOpenAi

Use the OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

OpenAiCompatible DestinationSnowflakeCortexConfigurationEmbeddingOpenAiCompatible

Use a service that's compatible with the OpenAI API to embed text.

AzureOpenAi DestinationSnowflakeCortexConfigurationEmbeddingAzureOpenAi

Use the Azure-hosted OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

Cohere DestinationSnowflakeCortexConfigurationEmbeddingCohere

Use the Cohere API to embed text.

Fake DestinationSnowflakeCortexConfigurationEmbeddingFake

Use a fake embedding made out of random vectors with 1536 embedding dimensions. This is useful for testing the data pipeline without incurring any costs.

OpenAi DestinationSnowflakeCortexConfigurationEmbeddingOpenAi

Use the OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

OpenAiCompatible DestinationSnowflakeCortexConfigurationEmbeddingOpenAiCompatible

Use a service that's compatible with the OpenAI API to embed text.

azureOpenAi DestinationSnowflakeCortexConfigurationEmbeddingAzureOpenAi

Use the Azure-hosted OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

cohere DestinationSnowflakeCortexConfigurationEmbeddingCohere

Use the Cohere API to embed text.

fake DestinationSnowflakeCortexConfigurationEmbeddingFake

Use a fake embedding made out of random vectors with 1536 embedding dimensions. This is useful for testing the data pipeline without incurring any costs.

openAi DestinationSnowflakeCortexConfigurationEmbeddingOpenAi

Use the OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

openAiCompatible DestinationSnowflakeCortexConfigurationEmbeddingOpenAiCompatible

Use a service that's compatible with the OpenAI API to embed text.

azureOpenAi DestinationSnowflakeCortexConfigurationEmbeddingAzureOpenAi

Use the Azure-hosted OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

cohere DestinationSnowflakeCortexConfigurationEmbeddingCohere

Use the Cohere API to embed text.

fake DestinationSnowflakeCortexConfigurationEmbeddingFake

Use a fake embedding made out of random vectors with 1536 embedding dimensions. This is useful for testing the data pipeline without incurring any costs.

openAi DestinationSnowflakeCortexConfigurationEmbeddingOpenAi

Use the OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

openAiCompatible DestinationSnowflakeCortexConfigurationEmbeddingOpenAiCompatible

Use a service that's compatible with the OpenAI API to embed text.

azure_open_ai DestinationSnowflakeCortexConfigurationEmbeddingAzureOpenAi

Use the Azure-hosted OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

cohere DestinationSnowflakeCortexConfigurationEmbeddingCohere

Use the Cohere API to embed text.

fake DestinationSnowflakeCortexConfigurationEmbeddingFake

Use a fake embedding made out of random vectors with 1536 embedding dimensions. This is useful for testing the data pipeline without incurring any costs.

open_ai DestinationSnowflakeCortexConfigurationEmbeddingOpenAi

Use the OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

open_ai_compatible DestinationSnowflakeCortexConfigurationEmbeddingOpenAiCompatible

Use a service that's compatible with the OpenAI API to embed text.

azureOpenAi Property Map

Use the Azure-hosted OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

cohere Property Map

Use the Cohere API to embed text.

fake Property Map

Use a fake embedding made out of random vectors with 1536 embedding dimensions. This is useful for testing the data pipeline without incurring any costs.

openAi Property Map

Use the OpenAI API to embed text. This option is using the text-embedding-ada-002 model with 1536 embedding dimensions.

openAiCompatible Property Map

Use a service that's compatible with the OpenAI API to embed text.

ApiBase string

The base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

Deployment string

The deployment for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

OpenaiKey string

The API key for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

ApiBase string

The base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

Deployment string

The deployment for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

OpenaiKey string

The API key for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

apiBase String

The base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

deployment String

The deployment for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

openaiKey String

The API key for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

apiBase string

The base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

deployment string

The deployment for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

openaiKey string

The API key for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

api_base str

The base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

deployment str

The deployment for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

openai_key str

The API key for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

apiBase String

The base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

deployment String

The deployment for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

openaiKey String

The API key for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource

CohereKey string

CohereKey string

cohereKey String

cohereKey string

cohere_key str

cohereKey String

OpenaiKey string

OpenaiKey string

openaiKey String

openaiKey string

openai_key str

openaiKey String

BaseUrl string

The base URL for your OpenAI-compatible service

Dimensions double

The number of dimensions the embedding model is generating

ApiKey string

Default: ""

ModelName string

The name of the model to use for embedding. Default: "text-embedding-ada-002"

BaseUrl string

The base URL for your OpenAI-compatible service

Dimensions float64

The number of dimensions the embedding model is generating

ApiKey string

Default: ""

ModelName string

The name of the model to use for embedding. Default: "text-embedding-ada-002"

baseUrl String

The base URL for your OpenAI-compatible service

dimensions Double

The number of dimensions the embedding model is generating

apiKey String

Default: ""

modelName String

The name of the model to use for embedding. Default: "text-embedding-ada-002"

baseUrl string

The base URL for your OpenAI-compatible service

dimensions number

The number of dimensions the embedding model is generating

apiKey string

Default: ""

modelName string

The name of the model to use for embedding. Default: "text-embedding-ada-002"

base_url str

The base URL for your OpenAI-compatible service

dimensions float

The number of dimensions the embedding model is generating

api_key str

Default: ""

model_name str

The name of the model to use for embedding. Default: "text-embedding-ada-002"

baseUrl String

The base URL for your OpenAI-compatible service

dimensions Number

The number of dimensions the embedding model is generating

apiKey String

Default: ""

modelName String

The name of the model to use for embedding. Default: "text-embedding-ada-002"

Credentials DestinationSnowflakeCortexConfigurationIndexingCredentials

Database string

Enter the name of the database that you want to sync data into

DefaultSchema string

Enter the name of the default schema

Host string

Enter the account name you want to use to access the database. This is usually the identifier before .snowflakecomputing.com

Role string

Enter the role that you want to use to access Snowflake

Username string

Enter the name of the user you want to use to access the database

Warehouse string

Enter the name of the warehouse that you want to use as a compute cluster

Credentials DestinationSnowflakeCortexConfigurationIndexingCredentials

Database string

Enter the name of the database that you want to sync data into

DefaultSchema string

Enter the name of the default schema

Host string

Enter the account name you want to use to access the database. This is usually the identifier before .snowflakecomputing.com

Role string

Enter the role that you want to use to access Snowflake

Username string

Enter the name of the user you want to use to access the database

Warehouse string

Enter the name of the warehouse that you want to use as a compute cluster

credentials DestinationSnowflakeCortexConfigurationIndexingCredentials

database String

Enter the name of the database that you want to sync data into

defaultSchema String

Enter the name of the default schema

host String

Enter the account name you want to use to access the database. This is usually the identifier before .snowflakecomputing.com

role String

Enter the role that you want to use to access Snowflake

username String

Enter the name of the user you want to use to access the database

warehouse String

Enter the name of the warehouse that you want to use as a compute cluster

credentials DestinationSnowflakeCortexConfigurationIndexingCredentials

database string

Enter the name of the database that you want to sync data into

defaultSchema string

Enter the name of the default schema

host string

Enter the account name you want to use to access the database. This is usually the identifier before .snowflakecomputing.com

role string

Enter the role that you want to use to access Snowflake

username string

Enter the name of the user you want to use to access the database

warehouse string

Enter the name of the warehouse that you want to use as a compute cluster

credentials DestinationSnowflakeCortexConfigurationIndexingCredentials

database str

Enter the name of the database that you want to sync data into

default_schema str

Enter the name of the default schema

host str

Enter the account name you want to use to access the database. This is usually the identifier before .snowflakecomputing.com

role str

Enter the role that you want to use to access Snowflake

username str

Enter the name of the user you want to use to access the database

warehouse str

Enter the name of the warehouse that you want to use as a compute cluster

credentials Property Map

database String

Enter the name of the database that you want to sync data into

defaultSchema String

Enter the name of the default schema

host String

Enter the account name you want to use to access the database. This is usually the identifier before .snowflakecomputing.com

role String

Enter the role that you want to use to access Snowflake

username String

Enter the name of the user you want to use to access the database

warehouse String

Enter the name of the warehouse that you want to use as a compute cluster

Password string

Enter the password you want to use to access the database

Password string

Enter the password you want to use to access the database

password String

Enter the password you want to use to access the database

password string

Enter the password you want to use to access the database

password str

Enter the password you want to use to access the database

password String

Enter the password you want to use to access the database

ChunkSize double

Size of chunks in tokens to store in vector store (make sure it is not too big for the context if your LLM)

ChunkOverlap double

Size of overlap between chunks in tokens to store in vector store to better capture relevant context. Default: 0

FieldNameMappings List<DestinationSnowflakeCortexConfigurationProcessingFieldNameMapping>

List of fields to rename. Not applicable for nested fields, but can be used to rename fields already flattened via dot notation.

MetadataFields List<string>

List of fields in the record that should be stored as metadata. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered metadata fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array. When specifying nested paths, all matching values are flattened into an array set to a field named by the path.

TextFields List<string>

List of fields in the record that should be used to calculate the embedding. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered text fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array.

TextSplitter DestinationSnowflakeCortexConfigurationProcessingTextSplitter

Split text fields into chunks based on the specified method.

ChunkSize float64

Size of chunks in tokens to store in vector store (make sure it is not too big for the context if your LLM)

ChunkOverlap float64

Size of overlap between chunks in tokens to store in vector store to better capture relevant context. Default: 0

FieldNameMappings []DestinationSnowflakeCortexConfigurationProcessingFieldNameMapping

List of fields to rename. Not applicable for nested fields, but can be used to rename fields already flattened via dot notation.

MetadataFields []string

List of fields in the record that should be stored as metadata. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered metadata fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array. When specifying nested paths, all matching values are flattened into an array set to a field named by the path.

TextFields []string

List of fields in the record that should be used to calculate the embedding. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered text fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array.

TextSplitter DestinationSnowflakeCortexConfigurationProcessingTextSplitter

Split text fields into chunks based on the specified method.

chunkSize Double

Size of chunks in tokens to store in vector store (make sure it is not too big for the context if your LLM)

chunkOverlap Double

Size of overlap between chunks in tokens to store in vector store to better capture relevant context. Default: 0

fieldNameMappings List<DestinationSnowflakeCortexConfigurationProcessingFieldNameMapping>

List of fields to rename. Not applicable for nested fields, but can be used to rename fields already flattened via dot notation.

metadataFields List<String>

List of fields in the record that should be stored as metadata. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered metadata fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array. When specifying nested paths, all matching values are flattened into an array set to a field named by the path.

textFields List<String>

List of fields in the record that should be used to calculate the embedding. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered text fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array.

textSplitter DestinationSnowflakeCortexConfigurationProcessingTextSplitter

Split text fields into chunks based on the specified method.

chunkSize number

Size of chunks in tokens to store in vector store (make sure it is not too big for the context if your LLM)

chunkOverlap number

Size of overlap between chunks in tokens to store in vector store to better capture relevant context. Default: 0

fieldNameMappings DestinationSnowflakeCortexConfigurationProcessingFieldNameMapping[]

List of fields to rename. Not applicable for nested fields, but can be used to rename fields already flattened via dot notation.

metadataFields string[]

List of fields in the record that should be stored as metadata. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered metadata fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array. When specifying nested paths, all matching values are flattened into an array set to a field named by the path.

textFields string[]

List of fields in the record that should be used to calculate the embedding. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered text fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array.

textSplitter DestinationSnowflakeCortexConfigurationProcessingTextSplitter

Split text fields into chunks based on the specified method.

chunk_size float

Size of chunks in tokens to store in vector store (make sure it is not too big for the context if your LLM)

chunk_overlap float

Size of overlap between chunks in tokens to store in vector store to better capture relevant context. Default: 0

field_name_mappings Sequence[DestinationSnowflakeCortexConfigurationProcessingFieldNameMapping]

List of fields to rename. Not applicable for nested fields, but can be used to rename fields already flattened via dot notation.

metadata_fields Sequence[str]

List of fields in the record that should be stored as metadata. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered metadata fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array. When specifying nested paths, all matching values are flattened into an array set to a field named by the path.

text_fields Sequence[str]

List of fields in the record that should be used to calculate the embedding. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered text fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array.

text_splitter DestinationSnowflakeCortexConfigurationProcessingTextSplitter

Split text fields into chunks based on the specified method.

chunkSize Number

Size of chunks in tokens to store in vector store (make sure it is not too big for the context if your LLM)

chunkOverlap Number

Size of overlap between chunks in tokens to store in vector store to better capture relevant context. Default: 0

fieldNameMappings List<Property Map>

List of fields to rename. Not applicable for nested fields, but can be used to rename fields already flattened via dot notation.

metadataFields List<String>

List of fields in the record that should be stored as metadata. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered metadata fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array. When specifying nested paths, all matching values are flattened into an array set to a field named by the path.

textFields List<String>

List of fields in the record that should be used to calculate the embedding. The field list is applied to all streams in the same way and non-existing fields are ignored. If none are defined, all fields are considered text fields. When specifying text fields, you can access nested fields in the record by using dot notation, e.g. user.name will access the name field in the user object. It's also possible to use wildcards to access all fields in an object, e.g. users.*.name will access all names fields in all entries of the users array.

textSplitter Property Map

Split text fields into chunks based on the specified method.

FromField string

The field name in the source

ToField string

The field name to use in the destination

FromField string

The field name in the source

ToField string

The field name to use in the destination

fromField String

The field name in the source

toField String

The field name to use in the destination

fromField string

The field name in the source

toField string

The field name to use in the destination

from_field str

The field name in the source

to_field str

The field name to use in the destination

fromField String

The field name in the source

toField String

The field name to use in the destination

ByMarkdownHeader DestinationSnowflakeCortexConfigurationProcessingTextSplitterByMarkdownHeader

Split the text by Markdown headers down to the specified header level. If the chunk size fits multiple sections, they will be combined into a single chunk.

ByProgrammingLanguage DestinationSnowflakeCortexConfigurationProcessingTextSplitterByProgrammingLanguage

Split the text by suitable delimiters based on the programming language. This is useful for splitting code into chunks.

BySeparator DestinationSnowflakeCortexConfigurationProcessingTextSplitterBySeparator

Split the text by the list of separators until the chunk size is reached, using the earlier mentioned separators where possible. This is useful for splitting text fields by paragraphs, sentences, words, etc.

ByMarkdownHeader DestinationSnowflakeCortexConfigurationProcessingTextSplitterByMarkdownHeader

Split the text by Markdown headers down to the specified header level. If the chunk size fits multiple sections, they will be combined into a single chunk.

ByProgrammingLanguage DestinationSnowflakeCortexConfigurationProcessingTextSplitterByProgrammingLanguage

Split the text by suitable delimiters based on the programming language. This is useful for splitting code into chunks.

BySeparator DestinationSnowflakeCortexConfigurationProcessingTextSplitterBySeparator

Split the text by the list of separators until the chunk size is reached, using the earlier mentioned separators where possible. This is useful for splitting text fields by paragraphs, sentences, words, etc.

byMarkdownHeader DestinationSnowflakeCortexConfigurationProcessingTextSplitterByMarkdownHeader

Split the text by Markdown headers down to the specified header level. If the chunk size fits multiple sections, they will be combined into a single chunk.

byProgrammingLanguage DestinationSnowflakeCortexConfigurationProcessingTextSplitterByProgrammingLanguage

Split the text by suitable delimiters based on the programming language. This is useful for splitting code into chunks.

bySeparator DestinationSnowflakeCortexConfigurationProcessingTextSplitterBySeparator

Split the text by the list of separators until the chunk size is reached, using the earlier mentioned separators where possible. This is useful for splitting text fields by paragraphs, sentences, words, etc.

byMarkdownHeader DestinationSnowflakeCortexConfigurationProcessingTextSplitterByMarkdownHeader

Split the text by Markdown headers down to the specified header level. If the chunk size fits multiple sections, they will be combined into a single chunk.

byProgrammingLanguage DestinationSnowflakeCortexConfigurationProcessingTextSplitterByProgrammingLanguage

Split the text by suitable delimiters based on the programming language. This is useful for splitting code into chunks.

bySeparator DestinationSnowflakeCortexConfigurationProcessingTextSplitterBySeparator

Split the text by the list of separators until the chunk size is reached, using the earlier mentioned separators where possible. This is useful for splitting text fields by paragraphs, sentences, words, etc.

by_markdown_header DestinationSnowflakeCortexConfigurationProcessingTextSplitterByMarkdownHeader

Split the text by Markdown headers down to the specified header level. If the chunk size fits multiple sections, they will be combined into a single chunk.

by_programming_language DestinationSnowflakeCortexConfigurationProcessingTextSplitterByProgrammingLanguage

Split the text by suitable delimiters based on the programming language. This is useful for splitting code into chunks.

by_separator DestinationSnowflakeCortexConfigurationProcessingTextSplitterBySeparator

Split the text by the list of separators until the chunk size is reached, using the earlier mentioned separators where possible. This is useful for splitting text fields by paragraphs, sentences, words, etc.

byMarkdownHeader Property Map

Split the text by Markdown headers down to the specified header level. If the chunk size fits multiple sections, they will be combined into a single chunk.

byProgrammingLanguage Property Map

Split the text by suitable delimiters based on the programming language. This is useful for splitting code into chunks.

bySeparator Property Map

Split the text by the list of separators until the chunk size is reached, using the earlier mentioned separators where possible. This is useful for splitting text fields by paragraphs, sentences, words, etc.

SplitLevel double

Level of markdown headers to split text fields by. Headings down to the specified level will be used as split points. Default: 1

SplitLevel float64

Level of markdown headers to split text fields by. Headings down to the specified level will be used as split points. Default: 1

splitLevel Double

Level of markdown headers to split text fields by. Headings down to the specified level will be used as split points. Default: 1

splitLevel number

Level of markdown headers to split text fields by. Headings down to the specified level will be used as split points. Default: 1

split_level float

Level of markdown headers to split text fields by. Headings down to the specified level will be used as split points. Default: 1

splitLevel Number

Level of markdown headers to split text fields by. Headings down to the specified level will be used as split points. Default: 1

Language string

Split code in suitable places based on the programming language. must be one of ["cpp", "go", "java", "js", "php", "proto", "python", "rst", "ruby", "rust", "scala", "swift", "markdown", "latex", "html", "sol"]

Language string

Split code in suitable places based on the programming language. must be one of ["cpp", "go", "java", "js", "php", "proto", "python", "rst", "ruby", "rust", "scala", "swift", "markdown", "latex", "html", "sol"]

language String

Split code in suitable places based on the programming language. must be one of ["cpp", "go", "java", "js", "php", "proto", "python", "rst", "ruby", "rust", "scala", "swift", "markdown", "latex", "html", "sol"]

language string

Split code in suitable places based on the programming language. must be one of ["cpp", "go", "java", "js", "php", "proto", "python", "rst", "ruby", "rust", "scala", "swift", "markdown", "latex", "html", "sol"]

language str

Split code in suitable places based on the programming language. must be one of ["cpp", "go", "java", "js", "php", "proto", "python", "rst", "ruby", "rust", "scala", "swift", "markdown", "latex", "html", "sol"]

language String

Split code in suitable places based on the programming language. must be one of ["cpp", "go", "java", "js", "php", "proto", "python", "rst", "ruby", "rust", "scala", "swift", "markdown", "latex", "html", "sol"]

KeepSeparator bool

Whether to keep the separator in the resulting chunks. Default: false

Separators List<string>

List of separator strings to split text fields by. The separator itself needs to be wrapped in double quotes, e.g. to split by the dot character, use ".". To split by a newline, use "\n".

KeepSeparator bool

Whether to keep the separator in the resulting chunks. Default: false

Separators []string

List of separator strings to split text fields by. The separator itself needs to be wrapped in double quotes, e.g. to split by the dot character, use ".". To split by a newline, use "\n".

keepSeparator Boolean

Whether to keep the separator in the resulting chunks. Default: false

separators List<String>

List of separator strings to split text fields by. The separator itself needs to be wrapped in double quotes, e.g. to split by the dot character, use ".". To split by a newline, use "\n".

keepSeparator boolean

Whether to keep the separator in the resulting chunks. Default: false

separators string[]

List of separator strings to split text fields by. The separator itself needs to be wrapped in double quotes, e.g. to split by the dot character, use ".". To split by a newline, use "\n".

keep_separator bool

Whether to keep the separator in the resulting chunks. Default: false

separators Sequence[str]

List of separator strings to split text fields by. The separator itself needs to be wrapped in double quotes, e.g. to split by the dot character, use ".". To split by a newline, use "\n".

keepSeparator Boolean

Whether to keep the separator in the resulting chunks. Default: false

separators List<String>

List of separator strings to split text fields by. The separator itself needs to be wrapped in double quotes, e.g. to split by the dot character, use ".". To split by a newline, use "\n".