The United States is picking up signs of interest from the Taleban on the possibility of talks with Kabul to end their war after more than 16 years.
Last month, Ghani reached out to the Taliban offering talks without preconditions.
Mattis said, "There is interest that we've picked up from the Taliban side", Reuters reported.
But the Taliban has so far ruled out direct talks with the Western-backed government, which they say is illegitimate.
In a sign of Afghanistan's precarious security situation, reporters traveling with Mattis were, for the first time, not allowed to publish stories until after his party left the Kabul airport, according to The Associated Press. So the emphasis is on drawing in Taliban elements piecemeal.
The United States has in the past also expressed hope of peeling off elements of the Taliban and it was unclear how this new effort might be different.
Mattis comments should soften President Trump's stance on refusing to talk with Taliban who for decades have repeatedly terrorized people with violence. They have so far not responded to Ghani's overture.
As part of its new regional strategy announced in August 2017, Washington has stepped up assistance to the Afghan military in a bid to break the stalemate and force the militants to the negotiating table.
He said, "Of course it's all working to achieve a reconciliation".
But Taliban attacks have continued, including a January suicide bombing in Kabul that killed more than 100 people.
Former Trump Aide Sam Nunberg Arrives for DC Grand Jury
On Tuesday, Fox Business Network reported that Nunberg meant to seek treatment for alcoholism following his grand jury appearance. Grand jury proceedings are secretive affairs, typically conducted without a lawyer on hand during questioning.
In return, the militants would have to recognize the Kabul government and respect the rule of law. It marks Mattis' third visit to the country, where around 11,000 US troops are stationed.
The Afghan security forces were able to stop some attacks, Mattis said, though he noted he wanted to see them shift to a more "offensive mindset" in the coming months.
The Pentagon chief defined the final goal of the conflict as political reconciliation between the warring parties rather than a military victory.
While the United States has been stepping up battlefield pressure, Afghanistan's global partners have sought to build diplomatic support from neighboring countries to push the militants to the negotiating table.
He said: "Right now, we want the Afghans to lead, and provide the substance of the reconciliation effort".
"All wars come to an end", Mattis said.
United States intelligence officials are predicting the war will remain stalemated as the traditionally most intensive fighting season begins this spring.
"I want to talk to people here and see the reality of how they see it, and go back and talk to our intelligence agencies to get a full assessment of where we're at", he said.
The visit is Mattis's second since President Trump announced last August that, despite his instinct to pull US troops out of Afghanistan, his administration would take a more aggressive approach to the conflict, now in its 17th year.
These and other moves boosted the number of USA troops in Afghanistan by at least 3,500 to a total of more than 14,000.