The Liberal Democratic Party (LDP) proposed allowing five female lawmakers to observe its all-male board meetings.
They cannot talk during the meeting - only submit opinions afterwards.
Women in Japan have long been alienated from politics and economic participation.
The country ranked 121 in a list of 153 countries on the World Economic Forum's 2020 Global Gender Gap Index.
Two women currently sit on the 12-person board of the LDP, which has been in power almost continuously since 1955.
Toshihiro Nikai, the 82-year-old LDP secretary general, told a news conference on Tuesday that he wanted to bring a female perspective to the meetings.
He said he was aware of criticism of the male domination of the party's elected board and it was important that female members of the party "look" at the decision-making process, he was quoted by Reuters as saying.
"It is important to fully understand what kind of discussions are happening. Take a look, is what it is about," he said.
Japanese media reported that the five women would be allowed to sit in as observers on decision-making board meetings but would not be allowed to speak. They could submit their opinions to the secretariat office afterwards.
Currently just 46 of 465 politicians in Japan's Shugiin or House of Representatives are women - that is about 10%, compared to 25% global average.
By Mariko Oi, BBC News
As a Japanese woman, casual sexism is what I unfortunately got used to over the years. It may happen at business meetings, work drinks or family gatherings. At those moments, many of us just laugh, pretend we didn't hear it and move on.
That's why Mr Mori's comments didn't surprise me, and the governing party's decision to allow non-speaking women to attend their meetings is a tactic we're familiar with.
Under the previous administration of Shinzo Abe, the government set a target to increase the number of female leaders by 2020. When it didn't manage to hit the target, it quietly pushed back the deadline by a decade.
Critics have long argued that instead of trying to increase the number of women for the sake of it, what Japan needs is a fundamental change, from education to hiring practices.
Yoshiro Mori, 83, had remarked, in discussions over increasing the number of female board members, that "we have to make sure their speaking time is restricted somewhat - they have difficulty finishing."
Following a huge outcry, he resigned on Friday for what he called his "inappropriate remarks".
Major sponsors of the Tokyo Olympics criticised his comments, including major backer Toyota.
A group of female lawmakers wore white in a protest against his remarks, and Tokyo Governor Yuriko Koike said she would not attend a meeting of high-level Olympic officials.
Some 400 people were said to have withdrawn their applications to volunteer at the Olympic games, which are scheduled for later this year.
It doesn’t matter how many times you fail. It doesn’t matter how many times you almost get it right. No one is going to know or care about your failures, and neither should you. All you have to do is learn from them and those around you because all that matters in business is that you get it right once. Then everyone can tell you how lucky you are.