Does special research expire?

No. Special research does not expire, and functions as a story quest line. More info on Special Research

The information Nick provided answers your question to the extent that Special Research will not expire. However, with announcement of Celebi being open the public on August 20th, the latter half of your question is now answerable.

The Special Research will neither expire or replaced with other quests. You will be able to complete both Mew's and Celebii's Special Research simultaneously, as long as you have completed at least the 3rd phase of Mew's Special Research

You can begin completing Special Research on Celebi at any time as long as you have done the third group of tasks related to the Special Research on Mew, and actively finish tasks related to both at your own pace!