This is a brick constructed loop hole covering the steps leading up to the old battery at Western Place of Arms.  There was formerly a metal gate here, remains of which were still in place in 2015.