Colder air will invade this weekend with warmer temperatures returning next week