Markov Decision Processes And Discrete-Time Mean-Field Games Constrained With Costly Observations