Hello,
I’m under the assumption that you are asking about a target similar to make program-dfu. Unfortunately, without manually putting the device into DFU boot mode (i.e. holding the boot button and pressing reset button), command line reset is not really an option.
Alternatively you can use data packets received via USB to achieve something like this (see the USB CDC examples in seed for more details). Assuming that you have some message system in place, you could set up a message for reset and set some flag. Then you can periodically check for the state of this flag and call the existing software reset function (provided by the STM32 HAL library).
e.g.
#include "stm32h7xx_hal.h"
//...
// Reset message via received
if(ResetMessageRcvd)
HAL_NVIC_SystemReset();
Lastly, if you are asynchronously writing to any non-volatile memory at all in your software, you should definitely ensure that there are no writes pending before calling the HAL_NVIC_SystemReset function. Otherwise you run the risk of memory corruption.
Hopefully this information helps.