conflict of interest?

My boss re-assigned me to another location. Since my job requires it I have to sell my apartment in New Jersey with one bedroom and one bathroom, it has also an office space. I found an apartment in which I am interested in. However, the broker for this apartment is also the broker whom I wanted to sell my own apartment. Although, I have not signed any contract with him. I am wondering if its alright to be on his list. Should I avoid listing with the same firm because of a conflict of interest?