Unable to migrate short text list of items to new content type and reference it to source

[GuyLivni]

Expected Behavior

When migrating a field which is a List of items
I want to migrate each item to new entry
so that I can then reference each new entry item as a reference field in the source

Actual Behavior

Currently it is impossible to migrate a field using deriveLinkedEntries API, if the field is a "short text" list of items, and I want to create new entry for each item and then reference the items back to the source.

Example

from:

• product
  • product_ids (a, b, c) - short text, list
  • products (reference to the created newProduct types)

to:

• newProduct
  • product_id (a)
• newProduct
  • product_id (b)
• newProduct
  • product_id (c)

Possible Solution

Allow deriveLinkedEntries to create multiple entries from short list and reference it back.

Steps to Reproduce

1. Create a content with a short text, list of items.
2. Try to migrate and reference back each item of the list to new item using deriveLinkedEntries API

Context

The migration can only be done manually, by copying each field content.

Environment

• Node Version: v8.11.4
• Package Manager Version: 5.6.0
• Operating System: macOS Mojave
• Package Version: ^0.16.1

[Khaledgarbaya]

Hey @GuyLivni,

This is not straightforward with the migration but achievable.

First, you need to change the list field to a reference field that holds all the content. using deriveLinkedEntries from your list field to a reference field

Second, you use transformEntries to transform a single ref field to multiple ref field.

Please let me know if you have questions

[gkulasik-livongo]

@Khaledgarbaya can you elaborate a bit more here? I've got the same issue but am not fully understanding your question.

Building on @GuyLivni's example, how should change the list field to a reference field that holds the content, would this be a new intermediate content type?

Here is an illustrative example,

migration.deriveLinkedEntries({
            contentType: 'product',
            from: ['product_ids'],
            toReferenceField: 'products',
            derivedContentType: 'newProduct',
            derivedFields: ['product_id'],
            identityKey: async (from) => {
                if(from === undefined || from.product_ids === undefined) {
                    return;
                }
                // This is not right - need a way to work on each product_id from the list
                return from.product_ids['en-US'].toLowerCase().replace(' ', '-');
            },
            shouldPublish: true,
            deriveEntryForLocale: async (from, locale) => {
                if(from === undefined ||
                    from.product_ids === undefined ||
                    from.product_ids["en-US"] === undefined ||
                    from.answers["en-US"].length == 0 ||
                    locale !== 'en-US') {
                    return;
                }
                // Not right - need way to select the exact product id
                return {product_id: from.product_ids["en-US"]};
            }
        });

When I run a similar migration I get the following error with using 0.26.4 of Contentful CLI migration.

✖ Making requests (1/1048)
→ Unexpected token R in JSON at position 0

Stacktrace points too:
│ TypeError: Cannot read property 'forEach' of undefined │
│ at run (/usr/local/lib/node_modules/contentful-cli/node_modules/contentful-migration/built/bin/cli.js:156:24)

[Khaledgarbaya]

Hi @gkulasik-livongo,

the code will look something like this

// list to a ref field
const crypto = require('crypto')

module.exports = (migration) => {

  migration.deriveLinkedEntries({
    contentType: 'myCT',
    derivedContentType: 'referenceCT',
    from: ['myListField'],
    toReferenceField: 'myTempRefField',
    derivedFields: ['list'],// field name inside the referenceCT
    identityKey: async (fromFields) => {
      return crypto.createHmac('sha256', fromFields.content['en-US']).digest('hex')
    },
    shouldPublish: true,
    deriveEntryForLocale: async (inputFields, locale) => {
      if (locale !== 'en-US') {
        return;
      }
      const copy = inputFields.content[locale]
      const title = inputFields.title[locale]
      return {
        list
      };
    }
  })
}

this script will copy all the values from an ordinary list field to a reference field with a list field

    
module.exports = (migration) => {
  const article = migration.editContentType('article')  
  migration.transformEntries({
    contentType: 'myCT',
    from: ['myTempRefField'],
    to: ['mylistRefField'],
    transformEntryForLocale: (fromFields, currentLocale) => {
      if (!fromFields.copyRef) {
        return
      }
      return {products:fromFields.list.split('')} // assuming that mylistRef content type has a products field that's an array or refs
    }
  })

}

[gkulasik-livongo]

Hey @Khaledgarbaya, thanks for the help here!

I'm not sure I explained myself correctly. I don't see how the two snippets above would convert an array of strings into an array of objects. I've tried adapting your example to my use case but I am unable to access 'list' in the second snippet because it is a field on another content type - and all we get to access from the migration are links to the content, not the content itself.

Product might have been a bad example above, let me try with another example: Tags. Let's say I have a list of symbol (string) field called 'tags' on a 'Product' content type. Now instead of an array of string I want to convert tags to its own content type 'Tag' with one field 'tag_text'. So now Product will have a reference list field which contains many Tag content type objects, each with one field holding the tag_text (the original string). So, in essence, I am converting an array of strings to an array of objects. It is the same as converting a single text field to a single content type reference but just many to many (which is what I would expect this example to portray: https://github.com/contentful/contentful-migration/blob/master/examples/20-derive-entry-n-to-n.js).

This may require potentially just scripting this manually with the Content Management API.

[colbygarland]

Has there been any changes made for this? I am running into this exact use case as well, and would prefer to not have to manually migrate over the fields.

[mayakarabula]

This ticket has been closed due to inactivity. If you still need help resolving your issue, please reach out through our community Slack, or contact Contentful support directly.

[daviddelusenet]

Does someone actually got this to work?