Open data portals are fundamental tools for governments to achieve public transparency. Hence, several countries around the world have issued Freedom of Information (FOI) laws and acts, imposing that city administrations develop their own open data portals to publish local open data. However, many cities, specially in developing countries, lack the financial resources even to invest in basic IT services. These cities quite often do not have qualified people in open data best practices, transparency and IT solutions for constructing these portals, specially the middle and small-sized ones. There are many works pointing out guidelines and general problems in open data portals. However, there is a lack of an unified instrument for evaluating and suggesting good practices in constructing open data portals considering both open data portal requirements and FOI access laws and acts. In this work, we propose an instrument for evaluating open data portals that gathers functionalities present in platforms that support open data portals construction and recommendations from FOI access law. Specifically, we used recommendations from Brazilian Information Access Law, as it is an instance of FOI access recommendation. We evaluated and validated our instrument assessing portals of 26 Brazilian cities. Our case study shows that the instrument is effective for showing many problems in these portals, and also reveals the current scenario in these cities.