Drupal - How do I modify the "Formats" dropdown in CKEditor

I would use Styles instead of Formats, because styles are fully controlled by you, from admin interface, no need for editing files.

When you add it to the toolbar you can configure it and add as many elements as you wish.

Here is an example config:

You modify the "Formats" dropdown by changing the list of allowed HTML tags. Any block-level HTML tags are shown in the "Formats" dropdown by CKEditor automatically.

For a tutorial/full explanation, see https://www.drupaleasy.com/quicktips/limiting-block-level-styles-drupal-8-ckeditor.

You need to hook_editor_js_settings_alter(), for adding DIV, like this:

function YOUR_MODULE_editor_js_settings_alter(array &$settings) {
  foreach (array_keys($settings['editor']['formats']) as $text_format_id) {
    if ($settings['editor']['formats'][$text_format_id]['editor'] === 'ckeditor') {
      $settings['editor']['formats'][$text_format_id]['editorSettings']['format_tags'] .= ';div';
    }
  }
}