[source-hubspot] DuplicateKey error during stream refresh when custom object has same name as standard object

[jtruty]

Connector Name

source-hubspot

Connector Version

4.2.7

What step the error happened?

Configuring a new connector

Relevant information

Create a custom object in hubspot called "owners". Try to refresh the schema.

Relevant log output

Internal Server Error: Duplicate key io.airbyte.protocol.models.StreamDescriptor@417723ad[name=owners,namespace=<null>,additionalProperties={}] (attempted merging values io.airbyte.protocol.models.AirbyteStream@79d017cc[name=owners,jsonSchema={"$schema":"http://json-schema.org/draft-07/schema#","type":["null","object"],"properties":{"archived":{"type":["null","boolean"]},"lastName":{"type":["null","string"]},"createdAt":{"format":"date-time","type":["null","string"]},"firstName":{"type":["null","string"]},"teams":{"type":["null","array"],"items":{"type":"object","properties":{"name":{"type":["null","string"]},"id":{"type":["null","string"]},"membership":{"type":["null","string"]}}}},"id":{"type":["null","string"]},"userId":{"type":["null","integer"]},"email":{"type":["null","string"]},"updatedAt":{"format":"date-time","type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updatedAt],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}] and io.airbyte.protocol.models.AirbyteStream@4ad67f7c[name=owners,jsonSchema={"$schema":"http://json-schema.org/draft-07/schema#","additionalProperties":true,"type":["null","object"],"properties":{"archived":{"type":["null","boolean"]},"createdAt":{"format":"date-time","type":["null","string"]},"id":{"type":["null","string"]},"properties":{"type":["null","object"],"properties":{"hs_all_accessible_team_ids":{"type":["null","string"]},"hs_user_ids_of_all_notification_unfollowers":{"type":["null","string"]},"hs_all_team_ids":{"type":["null","string"]},"hs_read_only":{"type":["null","boolean"]},"hs_object_source_detail_1":{"type":["null","string"]},"hs_object_source_detail_3":{"type":["null","string"]},"hs_object_source_detail_2":{"type":["null","string"]},"hs_object_source_label":{"type":["null","string"]},"hubspot_team_id":{"type":["null","string"]},"hubspot_owner_id":{"type":["null","string"]},"hs_unique_creation_key":{"type":["null","string"]},"hs_object_source_id":{"type":["null","string"]},"hs_user_ids_of_all_notification_followers":{"type":["null","string"]},"hs_created_by_user_id":{"type":["null","number"]},"hs_createdate":{"format":"date-time","type":["null","string"]},"hs_was_imported":{"type":["null","boolean"]},"hs_all_owner_ids":{"type":["null","string"]},"owner_email":{"type":["null","string"]},"owner_name":{"type":["null","string"]},"hs_pinned_engagement_id":{"type":["null","number"]},"hs_user_ids_of_all_owners":{"type":["null","string"]},"hs_all_assigned_business_unit_ids":{"type":["null","string"]},"hs_lastmodifieddate":{"format":"date-time","type":["null","string"]},"hubspot_owner_assigneddate":{"format":"date-time","type":["null","string"]},"hs_object_id":{"type":["null","number"]},"hs_shared_user_ids":{"type":["null","string"]},"hs_object_source":{"type":["null","string"]},"hs_merged_object_ids":{"type":["null","string"]},"hs_shared_team_ids":{"type":["null","string"]},"hs_object_source_user_id":{"type":["null","number"]},"hs_updated_by_user_id":{"type":["null","number"]}}},"updatedAt":{"format":"date-time","type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updatedAt],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}])

Contribute

• Yes, I want to contribute

[marcosmarxm]

The problem arises because Hubspot stream already has a stream called owners. One strategy to fix this is to always append a prefix like custom_ to custom objects. @maxi297 can you take a look and give your opinion here as this probably will introduce breaking changes to the connector?

[jtruty]

The problem arises because Hubspot stream already has a stream called owners. One strategy to fix this is to always append a prefix like custom_ to custom objects. @maxi297 can you take a look and give your opinion here as this probably will introduce breaking changes to the connector?

Maybe we use the fully qualified object name for a custom object only if there is a conflict? Then I think we avoid breaking changes (since that case would never have worked).

[jtruty]

If this solution ☝️ sounds acceptable I can take a swing at a fix @maxi297 @marcosmarxm

[jtruty]

Added a PR to hopefully address this issue.