Lock mutex error

[GregCornis]

I'm trying to connect a Canon camera to a M350 via the PSDK. However, I'm struggling with the live view, with the following message

Encoding frame 5
Encoded with frameSize=34929
[21.517][utils]-[Error]-[DjiFlowController_SendData:330) mutex lock error
[21.517][channel]-[Error]-[DjiHighSpeedDataChannel_SendVideoStreamData:609) Send video stream error: 0x000000E3.
[21.517][user]-[Error]-[startMockLiveView:756) send video stream error: 0x000000E3.

I have the same error when I call DjiPayloadCamera_GetVideoStreamState

[47.718][channel]-[Error]-[DjiHighSpeedDataChannel_GetVideoStreamState:547) Lock mutex error.
[47.718][user]-[Info]-[startMockLiveView:715) Video state: busy 0 - 0 0 0

Here is basically the code that I'm running

// Task to send Live View data
void *startMockLiveView(void *ctx) {
  while (1) {
    // Get H264 NAL from somewhere else
    DjiPayloadCamera_SendVideoStream((const uint8_t *) nal->p_payload, size);
    usleep(100000);
  }
}


registerAbstractionHandlers();        // Register Osal & Nal & UART etc handlers 
DjiCore_Init(&userInfo);              // Correct info, the rest of the PSDK works,
DjiPayloadCamera_Init();
DjiPayloadCamera_RegCommonHandler(&mockHandler);
DjiCore_ApplicationStart();
T_DjiOsalHandler *handler = DjiPlatform_GetOsalHandler();
T_DjiTaskHandle *task;
handler->TaskCreate("liveview", startMockLiveView, 2048, NULL, task);

I'm seeing a similar issue in the PSDK demo :

[12.065][core]-[Info]-[DjiIdentityVerify_UpdatePolicy:482) Update dji sdk policy file successfully
[13.204][infor]-[Info]-[DjiAircraftInfo_GetAdapterSerialNumber:806) Get serial number of Skyport V2 or X-Port is 39CDKCS0010013
[13.205][core]-[Info]-[DjiCore_Init:174) Identify AircraftType = Matrice 350 RTK, MountPosition = Payload Port NO1, SdkAdapterType = Skyport V2
[13.387][time_sync]-[Info]-[DjiTimeSync_PushAppTimeHandle:111) Current time: 2024.10.25 17:45:55.
[22.391][cam]-[Error]-[DjiPayloadCamera_RegMediaDownloadPlaybackHandler:987) lock mutex error: 0x000000E3.
[22.391][user]-[Error]-[DjiTest_CameraEmuMediaStartService:199) psdk camera media function init error.
[22.391][user]-[Error]-[main:249) camera emu media init error

Do you know what is going wrong ? Thanks

[GregCornis]

Edit : I've added

const T_DjiDataChannelBandwidthProportionOfHighspeedChannel bandwidthProportionOfHighspeedChannel =
        {10, 60, 30};
DjiHighSpeedDataChannel_SetBandwidthProportion(bandwidthProportionOfHighspeedChannel);

Now I don't have the warning anymore, and can successfully call DjiPayloadCamera_GetVideoStreamState. However no image is shown on DJI Pilot

[GregCornis]

I've also tried to set the format to H264_CUSTOM but that fails, though I'm doing it before ApplicationStart()

[25.272][cam]-[Error]-[DjiPayloadCamera_SetVideoStreamType:799) remove negotiated device sync error: 0x000000E1.
[25.272][user]-[Error]-[main:1179) Set stream type error

I'm sending video as NAL decoded with libx264 (annex B enabled), so it should be spec-compliant.

[dji-dev]

Agent comment from Leon in Zendesk ticket #120335:

Hello, which version of PSDK are you using? If it is 3.9.1, it only supports setting DJI-H264 channel, but not Custom-H264. We suggest that you replace the H264 video file pushed in the sample to your own directory, use your code to push the sample file, and see if there is a black screen and whether it can be pushed successfully. This way, you can check whether there is a problem with your own code implementation.

°°°

[GregCornis]

Ok I didn't know that (I don't think it's written in the docs). So what is DJI-H264 ? It looks like it's NAL H264 concatenated with audio ({0x00, 0x00, 0x00, 0x01, 0x09, 0x10}) .
Also, are there specific H264 parameters to use ? Here is what I'm using

    // x264 param
    x264_param_default_preset(&param, "ultrafast", NULL)
    param.i_bitdepth = 8;
    param.i_csp = X264_CSP_I422;
    param.b_vfr_input = 0;
    param.b_repeat_headers = 1;
    param.b_annexb = 1;
    param.i_bframe = 0;
    param.b_cabac = 0;
    param.b_deblocking_filter = 0;
    param.i_bframe_adaptive = 0;
    param.analyse.b_transform_8x8 = 0;
    param.analyse.i_me_method = X264_ME_DIA;
    param.analyse.i_trellis = 0;
    param.rc.b_mb_tree = 0;
    param.i_sync_lookahead = 0;
    param.rc.i_lookahead = 0;

[dji-dev]

Agent comment from Leon in Zendesk ticket #120335:

Hello, sorry, this is our oversight. We did not inform developers of this issue in advance or publicly. We are currently fixing it and will soon be able to use two channels of streaming separately. We have given the standards for video streaming in the document, which are for DJI format and Custom format respectively. You can check here:
https://developer.dji.com/doc/payload-sdk-tutorial/en/model-instruction/payload-develop-criterion.html#video-stream-transmission-standard
In the code, you only need to pay attention to the setting of the function:
DjiPayloadCamera_SetVideoStreamType(DJI_CAMERA_VIDEO_STREAM_TYPE_H264_DJI_FORMAT);
s_cameraVideoStreamType = DJI_CAMERA_VIDEO_STREAM_TYPE_H264_DJI_FORMAT;

°°°

[GregCornis]

Hi, I can't manage to get it to work. I'm streaming the same video file as in your sample code (PSDK0005.h264), but somehow nothing is showing. I have set the format to DJI_FORMAT, and I'm sending the AUD frame after every NAL. Do you know if there is something missing ? I'm thinking maybe it's something related to the HighBandwithChannel, or some other configuration

[dji-dev]

Agent comment from Leon in Zendesk ticket #120335:

Hello, could you please provide us with the PSDK debug log for confirmation? Did your PSDK show any error message when the screen is black? We may be able to analyze and confirm from the error message.

°°°

[GregCornis]

Hi, sorry for the delay, here are the logs:

[0.005][core]-[Info]-[DjiCore_Init:106) Payload SDK Version : V3.9.1-beta.0-build.2090
[1.836][adapter]-[Info]-[DjiAccessAdapter_Init:231) Identify aircraft series is Matrice 300 Series
[1.836][adapter]-[Info]-[DjiAccessAdapter_Init:264) Identify mount position type is Payload Port Type
[2.030][adapter]-[Info]-[DjiAccessAdapter_Init:365) Identity uart0 baudrate is 460800 bps
[2.046][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[3.046][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[4.046][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[5.046][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[6.047][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[7.047][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[8.047][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[9.047][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[10.047][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[11.047][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[12.047][adapter]-[Info]-[DjiPayloadNegotiate_Init:243) Waiting payload negotiate finished.
[14.107][core]-[Info]-[DjiIdentityVerify_UpdatePolicy:474) Updating dji sdk policy file...
[15.107][core]-[Info]-[DjiIdentityVerify_UpdatePolicy:482) Update dji sdk policy file successfully
[16.184][infor]-[Info]-[DjiAircraftInfo_GetAdapterSerialNumber:806) Get serial number of Skyport V2 or X-Port is 39CDKCS0010013
[16.185][core]-[Info]-[DjiCore_Init:174) Identify AircraftType = Matrice 300 RTK, MountPosition = Payload Port NO1, SdkAdapterType = Skyport V2
[16.283][time_sync]-[Info]-[DjiTimeSync_PushAppTimeHandle:111) Current time: 2024.11.22 10:45:27.
DJI READY
[24.360][core]-[Info]-[DjiCore_ApplicationStart:238) Start dji sdk application
Starting mock live view
Press enter to finish
x264 [info]: using cpu capabilities: ARMv8 NEON
x264 [info]: profile High 4:2:2, level 5.1, 4:2:2, 8-bit
Starting loop
Seeking 0
Reading 37454
Sending frame 0 0 0 1 6 5 255 255
Time to encode: 39711 us
MockGetSystemState called
MockGetSDCardState called
MockGetMode called
MockGetPhotoTimeIntervalSettings called
MockGetPhotoBurstCount called
MockGetShootPhotoMode called
Seeking 37454
Reading 8888
Sending frame 0 0 0 1 65 154 38 34
Time to encode: 139806 us
Seeking 46342
Reading 1858
Sending frame 0 0 0 1 65 154 70 33
Time to encode: 239885 us
MockGetSystemState called
MockGetSDCardState called
MockGetMode called
MockGetPhotoTimeIntervalSettings called
MockGetPhotoBurstCount called
MockGetShootPhotoMode called
Seeking 48200
Reading 415
Sending frame 0 0 0 1 65 154 102 33
Time to encode: 339968 us
Seeking 48615
Reading 661
Sending frame 0 0 0 1 65 154 134 33
Time to encode: 440113 us
MockGetSystemState called

And that's the relevant code

int main() {
    // Handle Ctrl-C
    signal(SIGTERM, exitHandler);

    DjiErrorCode code = setupDJISDK();
    if (code != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        USER_LOG_ERROR("Couldn't setup DJI SDK");
        return 1;
    }

    printf("DJI READY\n");
    
    // Test LiveView

    T_DjiReturnCode ret = DjiPayloadCamera_Init();
    if (ret != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        USER_LOG_ERROR("Couldn't init camera");
        return 1;
    }

    const T_DjiDataChannelBandwidthProportionOfHighspeedChannel bandwidthProportionOfHighspeedChannel =
        {10, 60, 30};
    ret = DjiHighSpeedDataChannel_SetBandwidthProportion(bandwidthProportionOfHighspeedChannel);
    if (ret != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        USER_LOG_ERROR("Couldn't set high speed proportion");
        return 1;
    }
    DjiLowSpeedDataChannel_Init();


    T_DjiCameraCommonHandler mockHandler = {
        .GetSystemState = MockGetSystemState,
        .SetMode = MockSetMode,
        .GetMode = MockGetMode,
        .StartRecordVideo = MockStartRecordVideo,
        .StopRecordVideo = MockStopRecordVideo,
        .StartShootPhoto = MockStartShootPhoto,
        .StopShootPhoto = MockStopShootPhoto,
        .SetShootPhotoMode = MockSetShootPhotoMode,
        .GetShootPhotoMode = MockGetShootPhotoMode,
        .SetPhotoBurstCount = MockSetPhotoBurstCount,
        .GetPhotoBurstCount = MockGetPhotoBurstCount,
        .SetPhotoTimeIntervalSettings = MockSetPhotoTimeIntervalSettings,
        .GetPhotoTimeIntervalSettings = MockGetPhotoTimeIntervalSettings,
        .GetSDCardState = MockGetSDCardState,
        .FormatSDCard = MockFormatSDCard
    };

    ret = DjiPayloadCamera_RegCommonHandler(&mockHandler);
    if (ret != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        USER_LOG_ERROR("Couldn't register camera common handler");
        return 1;
    }

    ret = DjiPayloadCamera_SetVideoStreamType(DJI_CAMERA_VIDEO_STREAM_TYPE_H264_DJI_FORMAT);
    if (ret != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        USER_LOG_ERROR("Set stream type error");
        return DJI_ERROR_SYSTEM_MODULE_CODE_SYSTEM_ERROR;
    }

    ret = DjiCore_ApplicationStart();
    if (ret != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        USER_LOG_ERROR("start sdk application error");
        return DJI_ERROR_SYSTEM_MODULE_CODE_SYSTEM_ERROR;
    }
    
    // char ipAddr[DJI_IP_ADDR_STR_SIZE_MAX] = {0};
    // uint16_t port = 0;
    // ret = DjiPayloadCamera_GetVideoStreamRemoteAddress(ipAddr, &port);
    // if (ret == DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
    //     USER_LOG_INFO("Get video stream remote address: %s, port: %d", ipAddr, port);
    // } else {
    //     USER_LOG_ERROR("get video stream remote address error.");
    //     return 1;
    // }

    // Send video stream
    T_DjiTaskHandle *task;
    T_DjiOsalHandler *handler = DjiPlatform_GetOsalHandler();
    handler->TaskCreate("liveview", startMockLiveView, 2048, NULL, task);

    printf("Press enter to finish\n");
    std::cin.ignore();

    finish();
    printf("Done.\n");

    return 0;
}

[GregCornis]

I have tried the PSDK demo again, and it stopped working. I'm seeing the same behavior, with no liveview on DJI Pilot. I don't know if it's related but I have a warning in the HMS : "Gimbal 1 payload voltage too low". Is it because the UART should be connected with 5V ? It's currently using 3.3V as it is connected to a Raspberry Pi

[dji-dev]

Agent comment from Leon in Zendesk ticket #120335:

Hello, I'm sorry, this log is probably not what we need. Maybe you can refer to this:
How to export PSDK Debug log
Before pushing the video stream file, have you tried to replace your video stream file with the sample file and let the sample code send it? This can confirm whether your video stream file itself is abnormal or whether there is a problem with your push implementation. This is a quicker way.

°°°

[GregCornis]

Hi, yes I'm tried the sample file, and even the sample code. As I mentioned above it didn't work, and only shows a black screen on DJI Pilot.
Sorry I'm on GitHub and can't see your link for PSDK Debug Log

[dji-dev]

Agent comment from Leon in Zendesk ticket #120335:

Hi, we are sorry for the bad experience, you can export from here.
Please note that if you get a black screen even when using the sample program, you need to check the firmware version of your skyportV2. If it is used with M350, it should be version 1.3.5 or above.
https://sdk-forum.dji.net/hc/en-us/articles/38861383793945-How-to-export-PSDK-Debug-log

°°°