Exporting layers to different sheets of an Excel file

On QGIS 3.10.3 or above

You can do that in this way:

# Prepare options for layer1
options = QgsVectorFileWriter.SaveVectorOptions()
options.driverName = "XLSX"
options.fileEncoding = "utf-8"
options.layerOptions = ["GEOMETRY=AS_XYZ"]
options.layerName = 'layer1'

# Export layer1
result, error_str = QgsVectorFileWriter.writeAsVectorFormatV2(
    layer1,
    '/tmp/my_file.xlsx',
    QgsProject.instance().transformContext(),
    options
)
print(result)

# Prepare options for layer2
options.actionOnExistingFile = QgsVectorFileWriter.CreateOrOverwriteLayer
options.layerName = 'layer2'

# Export layer2
result, error_str = QgsVectorFileWriter.writeAsVectorFormatV2(
    layer2,
    '/tmp/my_file.xlsx',
    QgsProject.instance().transformContext(),
    options
)
print(result)

The key is to open the file in update mode when exporting the second layer, i.e., to set options.actionOnExistingFile = QgsVectorFileWriter.CreateOrOverwriteLayer. See the full list of options in the docs

---

On QGIS 3.10.2 or below

You can use QgsVectorFileWriter.writeAsVectorFormat() in this way:

# Export layer
result, error_str = QgsVectorFileWriter.writeAsVectorFormat(
    layer,
    '/tmp/my_file.xlsx',
    options  # See above how to set the options object
)