I found this information about an English-language rugby club running on Saturday mornings at Da An Park, 10-12. It's TWD200 per session per child, but if you love rugby and want your children to try something that's in English, it could be worth further investigation.
As we are in NZ for the World Cup build-up, my kids are getting into the idea of being a "Small Black". I was going to put my son into a team here but he, unfortunately, broke his arm on the third day of our trip. I'm not sure how enthusiasm for the game will hold up when we return to Taiwan.