Metabase ingestion is using an old dashboard URL

[7onn]

Describe the bug
Metabase ingestion is using an old dashboard URL. I'm running Metabase on v0.48.1 and for this version, the dashboard endpoint changed a little and it requires a /public added on the aforementioned URL in DataHub's code.

To Reproduce
Steps to reproduce the behavior:

1. Setup Metabase v0.48.1
2. Setup the latest version of DataHub
3. Add some dashboard with public sharing enabled
4. Run DataHub Ingestion sourcing it
5. See error

Expected behavior
I'd expect DataHub ingestions'code to be on sync with the source's modern version

Additional context
I'd be willing to contribute on the fix. I just want to understand if we need to "just fix it" or make it backward compatible e.g. by making the dashboard endpoint configurable in the source.

[evgeniybobok]

Metabase removed GET /api/dashboard from their API in 0.48.0 metabase/metabase#35235

To list all the dashboards using the API, third parties can still use GET /api/collection to get the list of collections, and for each collection use GET /api/collection/:id/items?models=dashboard, and finally GET /api/dashboard/:id to get a specific dashboard

[evgeniybobok]

And they are going to restore GET /api/dashboard for backwards compatibility, but

No guarantees will be made to keep it up-to-date with the officially supported alternative, which is to use GET /api/collection

Also, this is just a temporary fix: metabase/metabase#37455

We should restore it for the next release of 48, with a deprecation notice to remove in the next release.

[lopezco]

metabase team restored the dashboard endpoint on version v0.48.3 (metabase/metabase#37458)
However, I think the the API response changed format as the ordered_cards doesn't exist anymore replaced by dashcards.

A quick patch could be something like: d45c680

I still think that the ingestion via their new API should also be implemented.

Response from /api/dashboard/:id for metabase 0.48.3:

{
    "description": null,
    "archived": false,
    "collection_position": null,
    "dashcards": [
        {
            "size_x": 12,
            "dashboard_tab_id": null,
            "series": [],
            "action_id": null,
            "collection_authority_level": null,
            "card": {},
            "updated_at": "2024-01-22T14:15:36.572015Z",
            "col": 0,
            "id": 2,
            "parameter_mappings": [],
            "card_id": 2,
            "entity_id": "bH7A6whewZ_B8kEjrhbdD",
            "visualization_settings": {},
            "size_y": 6,
            "dashboard_id": 1,
            "created_at": "2024-01-22T12:23:41.999383Z",
            "row": 1
        },
        {
            "size_x": 24,
            "dashboard_tab_id": null,
            "series": [],
            "action_id": null,
            "collection_authority_level": null,
            "card": {},
            "updated_at": "2024-01-22T14:15:36.572015Z",
            "col": 0,
            "id": 3,
            "parameter_mappings": [],
            "card_id": null,
            "entity_id": "jOpjajMjXy5Z49rbeTE-C",
            "visualization_settings": {
                "virtual_card": {
                    "name": null,
                    "display": "heading",
                    "visualization_settings": {},
                    "dataset_query": {},
                    "archived": false
                },
                "dashcard.background": false,
                "text": "Instant Loans Overview"
            },
            "size_y": 1,
            "dashboard_id": 1,
            "created_at": "2024-01-22T14:15:36.572015Z",
            "row": 0
        }
    ],
    "param_values": null,
    "can_write": true,
    "tabs": [],
    "enable_embedding": false,
    "collection_id": 2,
    "show_in_getting_started": false,
    "name": "Overview",
    "caveats": null,
    "collection_authority_level": null,
    "creator_id": 1,
    "updated_at": "2024-01-22T14:15:36.572015Z",
    "made_public_by_id": null,
    "embedding_params": null,
    "cache_ttl": null,
    "id": 1,
    "position": null,
    "entity_id": "Fa4qmexfobQ3Jt7NK3tmB",
    "param_fields": null,
    "last-edit-info": {
        "id": 1,
        "email": "[email protected]",
        "first_name": "a",
        "last_name": "email",
        "timestamp": "2024-01-22T14:15:36.693897Z"
    },
    "collection": {
        "authority_level": null,
        "description": null,
        "archived": false,
        "slug": "instant_loans",
        "name": "Instant Loans",
        "personal_owner_id": null,
        "type": null,
        "id": 2,
        "entity_id": "E6ysOXBs9zxHPhPuvQTYe",
        "location": "/",
        "namespace": null,
        "is_personal": false,
        "created_at": "2024-01-22T10:59:15.907933Z"
    },
    "parameters": [],
    "auto_apply_filters": true,
    "created_at": "2024-01-22T10:59:31.6482Z",
    "public_uuid": null,
    "points_of_interest": null
}

[7onn]

@lopezco my PR is taking care of that: https://github.com/datahub-project/datahub/pull/9631/files#diff-f7b9a31becd260463b7d2a4670f5c1ce000be42f0a35fc4128cf9efce90fdf05R271