The Pentagon Papers answer those questions. Follow the links to the original documents.

• Discontent with the Diem regime became acute in August, 1963. An 8/21/63 State Department memo makes clear the issue was serious. Diem’s South Vietnamese government had stormed Buddhist pagodas on that day. Unresolved issues between the government and protesting Buddhist monks had continued since the spring of 1963, when several monks, including Quang Duc, set themselves on fire. Madame Nhu, the President’s sister-in-law, disparagingly called the monks "Buddhist Barbecues" and offered to give them more gasoline. The repressive action in August crystallized the thinking that "Diem must go."
  
  
• President Kennedy received a briefing on August 27, 1963. William Colby, "the Vietnam expert from CIA," related a discussion he had with two South Vietnamese generals the day before. "One general said the situation for a coup was favorable and forecast that one would take place within a week."
  
  
• In the same meeting, Ambassador Nolting observed that "the Vietnamese generals haven’t the guts of Diem or Nhu."
  
  
• Two days later, the President sent a private message to Henry Cabot Lodge, Jr., the U.S. ambassador to South Vietnam. The message makes it clear that JFK supported the plan of the generals but reserved the right, as Commander in Chief, to change his mind at the last minute.
  
  
• Insisting on utmost secrecy between himself and Cabot Lodge, the President instructed the Ambassador that no response was required but, if one were sent:
  
  

  ...your answer should be unnumbered and headed "For President Only, Pass White House directly, no other distribution whatsoever."

• As summer turned to fall, the President wanted to maintain strict secrecy and, according to McGeorge Bundy, to "limit public knowledge."
  
  
• A CIA memo advises that the "Tran Kim Tuyen Coup d’Etat Group" attempted to assassinate Nhu on September 13, 1963, but the plot "failed when the detonator of the explosive failed to work." Even so, the group was confident "the coup would eventually succeed." Targeted, among others, was "the Ngo family..."
  
  
• Four days after the failed plot, a memo from Saigon to Washington expresses concern that Nhu was "negotiating for settlement with North." That, of course, would have been contrary to U.S. interests in South Vietnam.
  
  
• A CIA report, prepared for the President and dated September 19, 1963, indicates that U.S. military and intelligence people in South Vietnam were divided over the coup issue. Page two of the memo makes an interesting point:
  
  

  The pro-coup CIA Group is considered most interested in putting a government in which they can control, and thus justify the immense sums they have already spent in promoting stillborn coup d’etats and in collecting misinformation.

• By October 12, intelligence reports advised that the South Vietnamese government might stage
  
  

  a "spontaneous” demonstration against the American embassy, possibly culminating in its sacking and/or the assassination of key officials, including Ambassador Henry Cabot Lodge, Jr. These reports are difficult to evaluate, their sourcing is hazy, and it is possible that they have been deliberately started by the regime as a psychological warfare tactic, aimed at intimidating the United States and keeping the United States off balance. [The link is to page three of the report.]