Are you doing the count-off?  If not, how many beats are you given to set the tempo in your head?  Are you doing "1-2, 1-2-3-4" or something similar that will give you a little time to lock in?

Counting off is not as easy as alot of people think.  I still devote practice time to it on occassion.

You need a metronome or click device.  Most drummers don't work/practice with these tools, and most drummers are cheating themselves out of being the best they can be.

Dave suggests beginning the count with a "One- Two" He needs to reiterate that that should be counted in half time of the actual tempo.  So if your tempo marking is 1/4 note at 120 bpm you would only count the 1 (vocally stated as "ONE" and clicked simultaneously) and the 3 (as vocally stated as "TWO" and clicked off simultaneously); the actual 2 and 4 of this measure are silent.  

For the second measure of the count in you want to vocalize and click off all 4 beats or whatever time signature you are in (for now- there are many different ways of counting-  and at least three in 4/4 duple)

Now, hopefully you all have "jumped" into the tune.  

It helps to use the "tap" feature on your click device to get a bpm then write it down.  I usually practice clicking and playing to this by looking at the light then looking away.  When you look back a few bars later, you should be right on.

Remember, internalize the time in your "heart" and you will fine.  It gets easier with practice.

Yes.  Sorry.  Thanks for the clarification, Felix.

a clarification that is

I wear contacts when I play also

If I did that my band mates would probably hurl their guitars at me.... just joking  ...... kind of.

When I count off I do the usual one, two, one two three four!

I just hit the sticks together. Makes it seem like i'm really in control of the band...