Emotional highs and lows

I consider myself to be fairly level-headed and capable of managing my emotions well. This ability to compartmentalize is probably the quality that allows me to remain sane while living this double life. Without keeping my lives separate, things would simply get messy. My ever deepening feelings for A. have begun to undermine my capacity for compartmentalism. On days when we don't see each other, A. and I usually keep in touch with a call or texts throughout our day. Even on busy days, I will text him in the morning and wish him an amazing day and we will text once more at night saying how much we miss one another in the evenings. I have come to expect this regular communication with A. I used to say that expectations are the worst things to have in an affair because one will without a doubt be disappointed. Disappointed I was yesterday when A. didn't text me all day. We had a day planned for today and he knows how annoyed I get at making plans last minute and still, radio silence all day long. Eventually, I texted him asking him how his day was and got a brief reply and then nothing. I got extremely annoyed and in the evening, I asked him to tell me if and at what time we could meet today so that I could plan my day. I have a life and it annoys me when A. just assumes that he can tell me in the morning to be somewhere and that that won't be a problem for me. 

I know that these things are small and silly. I also know that if A. were like my other lovers, these things wouldn't bother me at all. Because I am so emotionally involved in this affair, I constantly feel like I am on a series of highs and lows. This roller coaster of emotions is too much for me to handle. I have a husband, a career and a family who all demand so much of my energy and I feel spent because of how much of that energy I am giving to my relationship with A. 

I am meeting A. in two hours. I am sure he senses this shift in my mood. I don't want to be one of those mistresses who play mind games... I do not want to trouble him with how emotionally draining our relationship is to me.... But I honestly don't know how much longer I can do this. I need calm and peace in my life (although I know that calm and peace bore me quite quickly!). 

I feel like I am losing my mind. This is the trend in my relationship with A. Why does it feel so complicated? Is it worth it? What am I truly getting out of this?