You cannot change the initial group.
For "Group Week view" and "Group Day view", the screens show members of "Priority organization".
Once the group is shown, you can change it to any groups.

Alternatively, you can show the desired groups by placing Scheduler (Group Week View) portlet or Scheduler (Group Day View) portlet in portals.
For details of how to set up My Portal, refer to Configuring Portlets in My Portal.