Location-based Social Networking (LBSN) platforms have become increasingly popular in recent times to explore point-of-interest (POI) locations in groups. Recommending POIs to user groups is challenging due to conflicting preferences among group members. Existing algorithms for group recommendations focus on extracting the POI preferences of each group member using explicit features and propose aggregation functions to compute group relevance. However, these methods often overlook domain-specific latent information and the dynamics of group formation. To address these issues, we propose a novel Knowledge-based Context-Aware Group Recommender System (KCGRS). In the first stage, KCGRS utilizes a knowledge graph to learn domain-aware user and POI embedding by using user interaction and meta information. The knowledge embeddings of users and POIs are further infused with visit context in the second stage by using a feed-forward transformer. Finally, at the group level, KCGRS learns the group embedding as a weighted aggregate of context-infused embedding of group members. In the recommendation stage, KCGRS assumes the group to be hypothetical users for providing optimum recommendations. Experiments on the real-world Yelp dataset demonstrate that KCGRS outperforms seven state-of-the-art baselines with a 14.15% increase in Hit ratio and a 13.07% improvement in NDCG compared to the state of the art methods.