If you have an open fire it is most likely to be parging which is the old dry cement that used to line the chimney originally. This does not indicate anything is wrong structurally and happens a lot. The parging will land in your fireplace ready to swept away.
