How to assert if a std::mutex is locked?

Strictly speaking, the question was about checking the lockedness of std::mutex directly. However, if encapsulating it in a new class is allowed, it's very easy to do so:

class mutex :
    public std::mutex
{
public:
#ifndef NDEBUG
    void lock()
    {
        std::mutex::lock();
        m_holder = std::this_thread::get_id(); 
    }
#endif // #ifndef NDEBUG

#ifndef NDEBUG
    void unlock()
    {
        m_holder = std::thread::id();
        std::mutex::unlock();
    }
#endif // #ifndef NDEBUG

#ifndef NDEBUG
    /**
    * @return true iff the mutex is locked by the caller of this method. */
    bool locked_by_caller() const
    {
        return m_holder == std::this_thread::get_id();
    }
#endif // #ifndef NDEBUG

private:
#ifndef NDEBUG
    std::atomic<std::thread::id> m_holder;
#endif // #ifndef NDEBUG
};

Note the following:

1. In release mode, this has zero overhead over std::mutex except possibly for construction/destruction (which is a non-issue for mutex objects).
2. The m_holder member is only accessed between taking the mutex and releasing it. Thus the mutex itself serves as the mutex of m_holder. With very weak assumptions on the type std::thread::id, locked_by_caller will work correctly.
3. Other STL components, e.g., std::lock_guard are templates, so they work well with this new class.

std::unique_lock<L> has owns_lock member function (equivalent of is_locked as you say).

std::mutex gmtx;
std::unique_lock<std::mutex> glock(gmtx, std::defer_lock);

void alpha(void) {
   std::lock_guard<decltype(glock)> g(glock);
   beta(void);
   // some other work
}
void beta(void) {
   assert(glock.owns_lock()); // or just assert(glock);
   // some real work
}

EDIT: In this solution, all lock operations should be performed via unique_lock glock not 'raw' mutex gmtx. For example, alpha member function is rewritten with lock_guard<unique_lock<mutex>> (or simply lock_guard<decltype(glock)>).

You could just use a recursive_mutex, which can be locked multiple times on the same thread. Note: If it were my code, I would restructure it so that I don't need a recursive_mutex, but it will address your problem.