Is #include <Windows.h> bad practice?

Msdn documentation explicitly tells you (a) in which header file a function is declared and (b) which header file you are supposed to include.

Most functions tell you to include windows.h, for example SendMessage

Some function, that were added later or have very specific use cases, are only available through other header files, for example SetupDiEnumDeviceInfo.

So no, it is not bad practice to follow their advice. However, I strongly recommend disabling some parts of it before including via macro, e.g.

#define NOMINMAX
#include <Windows.h>

because otherwise you will get a min and a max macro that will interfere with std::min and std::max.

bits/stdc++.h is a compiler-specific header. If you compile your code with a different compiler it may or may not exist. It's not part of standard C++. It's a shortcut for access to the C++ standard library.

windows.h is an operating-system specific header. If you're compiling for Windows you need it, and every compiler that supports Windows will be okay with it. It doesn't necessarily come with the compiler; it comes with the Windows Software Development Kit. It provides access to the Windows API.

You might be able fine-tune your #include directives under Windows, but Windows is such a mess that unless you really want to spend time digging into it, you're better off just using windows.h.