Thanks for the recommendations
Had the same problem

AMOS - No, Freshdesk did nothing. It was in how I configured my end. The key is building  2 enterprise apps in Azure correctly, one for Agents and the other for contacts. One uses the built in Freshdesk app, the other a CUSTOM enterprise app.  Then having two Security profiles in Freshdesk. Each one pointing to a different enterprise app in Azure. I used the default profile for agents and created the custom one for contacts. I am assuming you could use the default one for contacts and the custom one for agents, but i already had them built and it's working so i didn't want to change them.  Make sure you also have people in your AAD allowed access to those enterprise apps in Azure or it will fail. 

I was able to successfully set up SSO for my Contacts using miniOrange SSO plugin for my WordPress site.

https://wordpress.org/plugins/miniorange-wp-as-saml-idp/

I have seen that these folks provide similar SSO solutions for platforms other than WordPress as well.

I totally recommend miniOrange if you need help with setting up SSO for your Freshdesk contacts or agents.

Thanks.

I ran into this as well. We had been using freshdesk SSO up until a month or so ago when we needed to make a cert change. When I did that, I was forced to switch to Freshworks SSO.  So I set this up, connecting OneLogin to Freshworks via Onelogin's built in connector and the Freshworks guide. It worked ok.

Then realized contacts couldn't login, only agents (this was not the case with the old freshdesk sso). So setup another OneLogin SAML connector for contacts, but it always errored out at login.

Eventually found this thread, and redid the agent connector to OneLogin using OneLogins generic SAML connector, rather than the freshworks one, and now all is working.