Yes you can temporarily or permanently stop the teller from using your account.