[SOLVED] Increasing performance on SCP

[stefanoostwegel]

Hi all,

I was diving in the documentation te see if we can find a way to speed up the SCP performance for incoming data.
It is suggested according to the example code to use the following c-store handler.

def handle_store(event):
    """Handle a C-STORE request event."""
    with open(event.request.AffectedSOPInstanceUID, 'wb') as f:
        # Write the preamble and prefix
        f.write(b'\x00' * 128)
        f.write(b'DICM')
        # Encode and write the File Meta Information
        write_file_meta_info(f, event.file_meta)
        # Write the encoded dataset
        f.write(event.request.DataSet.getvalue())

    # Return a 'Success' status
    return 0x0000

This is very interesting, since it could really help us out.
I cannot seem to understand how to implement the following two things.

1. How can i find the series UID without decoding/re-encoding the dataset?
2. How can i set the path to write the file to (we want to use the series UID to write in a specific directory), is that in python the f.write(event.request.DataSet.getvalue()) statement?

Thank you

[stefanoostwegel]

I now have the following implementation:

def handle_store(event):
    """Handle a C-STORE request event."""
    ds = event.dataset
    series_directory = settings.TEMP_FOLDER_SCP + ds.SeriesInstanceUID
    file = series_directory + '/' + event.request.AffectedSOPInstanceUID + '.dcm'


        with open(file, 'wb') as f:
            # Write the preamble, prefix and file meta information elements
            f.write(b'\x00' * 128)
            f.write(b'DICM')
            write_file_meta_info(f, event.file_meta)
            # Write the raw encoded dataset
            f.write(event.request.DataSet.getvalue())
        logger.debug('Saved file')

    # Return a 'Success' status
    return 0x0000

It is still arguable how much performance increase this has, since the dataset has to be encoded to get the seriesuid to create that directory.
If anyone has any suggestion to get the seriesUID more effectively, please let me know

[stefanoostwegel]

We have finally found a working solution to prevent opening the event.dataset, but still know where to write the data in the SCP.

When creating the c-move request, we create a custom message ID in the request:
It looks like this:

 responses = connection.send_c_move(dataset=self.ds,
                                         move_aet=self.remoteaet,
                                         query_model=PatientRootQueryRetrieveInformationModelMove,
                                         msg_id=int(self.move_originator_message_id))

This message ID can be found in the local SCP when the store request comes in.
It is found at

Move Originator Message ID	(0000,1031)	US	1	Contains the Message ID (0000,0110) of the C-MOVE-RQ Message from which this C-STORE sub-operations is being performed.

It can be read at https://dicom.nema.org/medical/dicom/current/output/chtml/part07/sect_9.3.html chapter 9.3.1.1

So to achieve our best performance during the c-move we de the following step:

1. Create a directory in the SCP folder with a unique number.
2. Set this number as message ID in the send_c_move request.
3. At the SCP, retrieve the 'Move Originator Message ID' from the header.
4. Set the SCP to write the files directly to it, if they do not exist yet.

Our SCP code now looks like this:

def storescp_handle_store(event):
    """
    The store SCP handler is optimized to store DICOM information without the encoding/decoding part.
    The event.dataset is not read, and the maximum PDU size is set to '0' to improve performance.
    See https://pydicom.github.io/pynetdicom/stable/examples/storage.html#storage-scp 'optimizing for speed'
    for more details.

    The Store handler will first test if the series directory exists. This makes sure that the incoming data is
    requested by DRM, and not unsolicited from any random application.
    Whenever the directory exists, the file should not exist. If the file is already on disk, we do not overwrite!
    Duplicates will be ignored and accepted by the handler.

    Whenever everything works out fine, the data is streamed into the file.
    :param event: PyNetDICOM event
    :return:
    """

    # event.request.MoveOriginatorMessageID this is the Message ID set in the c-move operation.
    series_directory = settings.TEMP_FOLDER_SCP + str(event.request.MoveOriginatorMessageID)
    file = series_directory + '/' + event.request.AffectedSOPInstanceUID + '.dcm'
    logger.debug(f'SCP stored file at {file}')
    logger.debug(f'original request id {str(event.request.MoveOriginatorMessageID)}')
    if not os.path.exists(series_directory):
        logger.debug('File received without a directory, we didnt expect the file so we abort the association')
        event.assoc.abort()
        return 0xC001

    # do not overwrite files in the directory. It should have been removed if we wanted a clean start.
    # Due to issues from a source it is possible to have more moves for a record then we hoped for.
    # Just skip whenever a duplicate exists
    if os.path.exists(file):
        logger.debug('Duplicate file received, do not overwrite')
        return 0x0000

    try:
        with open(file, 'wb') as f:
            # Write the preamble, prefix and file meta information elements
            f.write(b'\x00' * 128)
            f.write(b'DICM')
            write_file_meta_info(f, event.file_meta)
            # Write the raw encoded dataset
            f.write(event.request.DataSet.getvalue())
        logger.debug('Saved file')
        return 0x0000
    except Exception as e:
        logger.error(f'Storing file in SCP failed: {e}')
        return 0xC211

nb: this is part of a Python Dajngo project, hence the settings variables.