The reason why these topics appear so problematic to us is that we fail to differentiate between reality and appearance.
We all (presumably) believe in Advaita to some large degree even if we feel we have no direct knowledge of its truth. Thus we accept that reality is non-dual - there simply are not 'two things'. Nevertheless, we mistakenly perceive separate things out there and have what we deem to be separate thoughts and feelings about them. This is the world of 'appearance' and, though we probably accept this intellectually, the appearance seems all too real to us most of the time.
As far as the reality is concerned, it cannot be meaningful to talk about fate or free will. For there to be fate, there would have to be someone to whom separate things could happen or who could observe other things happening in some predetermined manner. For there to be free will, there would have to be someone who could choose between separate things. All of these ways of thinking belong to the illusory world of duality.
The words that we use in everyday language and the very way in which we think are part of this mistaken view. The concepts of space and time, cause and effect, are the mental constructs that we use in order to make sense of the appearance and they themselves bolster the dualistic view. Things have to be separated in either space or time (otherwise they would be the same thing). Cause and effect relationships must exist in order for us to be able meaningfully to connect events. The ways in which we think and the language that we use inevitably propagate this view and mean that we can never have objective knowledge of reality or talk about it in the ways that we normally use.
Thus, to reiterate, and to conclude the discussion of these topics at the level of 'reality', there is no fate or free will. There is no (satisfactory) answer because there is no question. How could 'I' have free will when 'I' am an illusion? The rest of the material below relates only to the mistaken world of appearances, where there appears to be someone who has a problem. It is how things appear to be but not how they actually are. In the context of the rope and snake metaphor, it is a discussion about how poisonous the snake is and what to do if you are bitten.
When we act, most of us would admit that we tend to 're-act' out of habit, rather than make a clear decision in the light of a clean observation of what is in front of us and what is needed. The habitual, automatic response obviously cannot involve much in the way of free will. If the mind is uncluttered with the usual junk of circling thoughts, however, is it not possible that, in the moment of observation, there is a choice and that a new and probably more appropriate response than the habitual one is realisable?
It is necessary that this view be upheld if the path of karma yoga is to have any meaning. This path advocates such 'aware' action in order to break down the habits and purify the mind. Automatic and selfishly motivated actions merely reinforce the patterns and mire us ever more deeply in the world of misery. The mechanism by which this occurs is that actions which are not simply responses to a need are said to generate effects in the future. If we do something that we believe to be wrong, we are very likely to experience feelings of guilt, for example, and this in turn may well affect our future behaviour.
Classically, some would go even further than this and say that our actions in the past actually bring about in some way the situations that we are presented with in the present, as though some god were giving us the opportunity to do better this time round. In this sense, of course the ideas relate to the concept of 'fate'. Again, classically, this ties in with the notion of reincarnation. If we live a wholly 'bad' life, we are very likely to come back as a cockroach and have to work our way through thousands of lives (again) until we finally have the privilege of being reborn as a man.
Free will implies choice. It means that we can act - we can choose and then we can act upon that choice. It assumes, therefore, that there is such a thing as cause and effect. Our choice becomes a cause for our effective action. All of this, then, is in contradiction of the idea of 'fate', which implies that something will happen regardless of our choices and actions; that an effect will occur, regardless of the causes that are in place. And yet, of course, those who believe in fate will argue that the fated outcome is a result of some (freely chosen) past action, possibly even in a previous incarnation.
The paradoxical claim of karma yoga is that an 'aware' choice in the moment somehow breaks down the chain of cause and effect resulting from past action and does not itself generate any new chain. Once all of this past 'karma' has been dissolved through 'right' action, enlightenment results. The sort of metaphor used to explain this is that of steering a boat against the current of a river. You cannot escape the influence of the current (past karma or fate) but you can navigate intelligently (exercise a degree of free will or choice).
For me, in the end, I rationalise it all through reflection on my own experience. It seems that all of the key decisions in my life have been essentially choiceless. Of course, you weigh up all of the pros and cons at the time but, having done so, you inevitably choose the most attractive option for you at the time. And whichever course of action does seem better depends upon all of our pre-conditioning by parents, education, reading, TV etc. Having thought about this subject for some time, I cannot imagine that anyone could ever convince me that they have any real free-will in any choice that they make.
Even at the most trivial level the same arguments apply. You might say that you are free to continue reading this or delete the message or throw the computer out of the window or any number of other variants. But analyse this. To begin with, only a few possibilities will occur to you. Which ones do occur will depend upon your personality, imagination and so on. Where do thoughts come from anyway? Can you choose to have a thought? Then, thoughts having arisen, the particular factors that determine your choice will depend upon other, already existing opinions, beliefs, preferences etc. Throwing the computer out of the window will probably not be seriously considered (unless you are very rich and particularly cavalier in your behaviour). Switching off or deleting may receive cursory consideration but will almost certainly be rejected - if you were sufficiently interested to begin reading the message, you will probably want to find out how it ends. So, even without knowing you personally, I can predict with a high degree of certainty that you will carry on (to the bitter end) - you simply do not have any choice!
You do not choose to be born or choose your parents (I know some may dispute this - Plato has his 'myth of Er' in the Republic, for example - but let them provide evidence!) so your genetic heritage is not in your control. Having been born into that environment, you cannot help but be conditioned by it - parents, education, religion, friends etc. are the initial source of all of your input. The results of these influences are the thoughts, opinions and beliefs that motivate your so-called free choices onward throughout your life. Would it, in any case, make any sense for actions not to be brought about by other events, thoughts etc.? We do not suddenly do something for no reason whatsoever. We are motivated by external situations or thoughts and respond to these according to our nature. It would make a nonsense of education and the legal system, for example, if people were not influenced in this way. In effect, we act in response to related causes. The discipline of karma yoga can replace the motivation of selfish desire by that of surrender of the fruits of action so that the being is gradually purified of habits and ignorance but this in no way alters the fact that the action is not 'ours' and is driven by simple cause and effect.
It is also possible to approach the subject by considering action itself. We really seem to believe it when we say 'I did such and such'. But a little thought shows that there is a virtually endless chain of events that lead up to any given action, no matter how trivial. 'I cleaned my teeth' simply involves a brush and toothpaste, doesn't it? But years of development, involving countless numbers of people have been involved in each of these objects. The plastic of the brush handle and toothpaste tube are processed from oil, again involving generations of discovery, research and invention. The oil itself is aeons old, the product of natural events. Carl Sagan said: "If you want to bake an apple pie from scratch, you must first create the universe."
Even at the final stage, where paste is applied to brush and brush to teeth, there is some doubt as to what exactly is going on. Of course, we feel that we are actually 'doing' something but is this really true? What do you actually do to cause the brain to trigger synaptic events and release hormones into the bloodstream and so on? It is possible to watch yourself 'doing something', such as walking or making a cup of tea, and to see that 'you' are not doing anything at all. It is happening of course but you are not initiating any of the events, they are simply taking pace within your awareness. And, what is even more interesting, is that when you allow actions to take place in this way, without any thought that you are doing something, they happen far more efficiently and beneficially than when you attempt to interfere.
The Kena Upanishad says: "By whom commanded and directed does the mind go towards its objects? Commanded by whom does the life force, the first cause move? At whose will do men utter speech? Which power directs the eye and the ear? It is the ear of the ear, the mind of the mind, the speech of the speech, the life of the life, the eye of the eye. . There the eye does not go, nor speech, nor mind. We do not know That; we do not understand how it can be taught. It is distinct from the known and also It is beyond the unknown."
Benjamin Libet carried out experiments to try to understand the nature of intentionality and motor action. What he did was to monitor the subject's brain in that region responsible for initiating action. The subject watches a clock on a computer screen. He is asked to press either button A or button B whenever he is ready and to note the time on the clock when he makes the conscious decision. The computer subsequently records the time verbally claimed for the decision and compares it with the time automatically recorded when the brain triggered the movement in the hand to press the button. The interesting thing is that the claimed decision to press a button always takes place after the brain starts to move the hand. Irrespective of whether or not this proves anything, our experience is effectively what we mean by the term free will. If what Libet concluded is true, however, what it means is that this 'feeling' of having free will is only a 'side effect' arising in the brain after some action has been initiated in automatic response to a combination of stimuli.
In today's society, we are constantly bombarded by claims that everything is ultimately explicable by science. Although consciousness itself is not yet fully understood, it is only a matter of time. Of course, as Advaitins, we probably accept that this is not so but the attitude is very insidious. The problem is that is the very nature of science to be objective. We, the observer-subject, devise hypotheses and carry out experiments on objects. This is what science is about. It ought not to be surprising then that, since the ultimate reality is non-dual, this method can never succeed when we are investigating reality itself. The mind can only operate in dualistic concepts. Wittgenstein summed up the position nicely: "Philosophers constantly see the method of science before their eyes and are irresistibly tempted to ask and to answer questions in the way science does. This tendency is the real source of metaphysics and leads philosophers into complete darkness."
Nisargadatta Maharaj said: "Destiny is usually thought of as something that is going to happen in the future. Destiny is all here-now. The script has already been written. I am playing all the characters in the movie; and I am witnessing the movie which is already done." I like Burt Harding's comment on this: "The movie of our life is being replayed like a movie-video, however, with one exception, we can either accept it or reject it as it is being replayed. Do you see the simplicity of it? When the ego, through ignorance, rejects the movie then we have suffering. If we accept the movie, we have enlightenment."
In the end then, free will has no meaning in respect of reality and only an illusory meaning as part of the phenomenal world. The Self is always free and, as Ralph Ellison said, "When I discover who I am, I'll be free". Unfortunately, as someone else said, "Before you can break out of prison, you must first realise you are locked up". The person is not free. Fortunately, the person does not exist.
In many ways, fully appreciating the essence of this topic is key to understanding our Self. Whilst we genuinely believe that we have free will, we continue to see ourselves as independent agents i.e. as separate and as 'doers' and 'enjoyers'. We are none of these things and an intellectual appreciation of that is a valuable first step towards liberation.