How to query Wikidata items using its labels?

Yes, you can search by label, for example:

SELECT distinct ?item ?itemLabel ?itemDescription WHERE{  
  ?item ?label "Something"@en.  
  ?article schema:about ?item .
  ?article schema:inLanguage "en" .
  ?article schema:isPartOf <https://en.wikipedia.org/>. 
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }    
}

see it on Query page.

As of today (June 2020), the best way to do this seems to be using these CirrusSearch extensions. The following does a substring search in all English labels and comes back with 10,000 results in <20 seconds. I believe it also searches in aliases and descriptions.

SELECT DISTINCT ?item ?label
WHERE
{
  SERVICE wikibase:mwapi
  {
    bd:serviceParam wikibase:endpoint "www.wikidata.org";
                    wikibase:api "Generator";
                    mwapi:generator "search";
                    mwapi:gsrsearch "inlabel:city"@en;
                    mwapi:gsrlimit "max".
    ?item wikibase:apiOutputItem mwapi:title.
  }
  ?item rdfs:label ?label. FILTER( LANG(?label)="en" )

  # … at this point, you have matching ?item(s) 
  # and can further restrict or use them
  # as in any other SPARQL query

  # Example: the following restricts the matches
  # to college towns (Q1187811) only

  ?item wdt:P31 wd:Q1187811 .
}

Link to this query

Following your question and the useful comments provided, I ended up with this query

SELECT ?item ?itemLabel
WHERE { 
  ?item rdfs:label ?itemLabel. 
  FILTER(CONTAINS(LCASE(?itemLabel), "city"@en)). 
} limit 10

For which I got those results

item          itemLabel
wd:Q515       city
wd:Q7930989   city
wd:Q15253706  city
wd:Q532039    The Eternal City
wd:Q1969820   The Eternal City
wd:Q3986838   The Eternal City
wd:Q7732543   The Eternal City
wd:Q7737016   The Golden City
wd:Q5119      capital city
wd:Q1555      Guatemala City

try it here