The best time to visit depends on your reasons for wanting come to the island.
Are you interested in birds, art, photography? Or perhaps you just want to get away from it all. If the latter then it's easy - any time is a good time! Although you may want to bear in mind that July and August are traditionally our busiest months so if you are looking for peace and quiet a visit in September or October is ideal.
Bardsey is a magical place for inspiration whether it be painting, writing or photography and it's difficult to say one time of year is any better than another but there are a few things to take into consideration.
Long days mean plenty of time to get out an about although getting that early morning light does mean an early start. Spring and summer is when the island is at its most colourful with fresh green grass and swathes of gorse and squill. It's also a good time if you're into macro photography - lots of insects and flowers to photograph.
The longer nights of autumn are great for dusk and night photography, there is much less light pollution than on the mainland (although it is still obvious in long exposures). the island looks a little browner now but you can get some glorious sky scapes (not restricted to autumn of course!). You may also be lucky enough to witness a storm, a special experience to see a wild sea crashing on to the rocks. the downside being your stay might be extended (or is that an upside?).