Stoke City's Erik Pieters in action with Arsenal's Shkodran Mustafi
The Spanish forward, a loan signing from Paris Saint-Germain, finished crisply in the 47th minute to bring Arsene Wenger's side down to earth after their opening 4-3 win over Leicester City.
It was an early blow to Arsenal's title aspirations and left them three points behind early pace-setters Manchester United, who won 4-0 for the second game running at Swansea City.
Jese, 24, only joined Stoke on Wednesday and he was quick to make an impression on the fans of his new team.
After driving at Arsenal's back four from halfway, he swapped passes with Saido Berahino and then drilled a left-foot shot past the advancing Petr Cech.
Arsenal, who remain without the injured Alexis Sanchez, thought they had equalised in the 72nd minute when Alexandre Lacazette lashed home, but it was chalked off due to a marginal - but correct - offside.
Stoke had not beaten Arsenal since December 2014.
Earlier, Romelu Lukaku had been on target again as Manchester United continued their fine form by crushing Swansea.
Eric Bailly, Paul Pogba and substitute Anthony Martial also found the net at the Liberty Stadium, giving Jose Mourinho's side their second big win after last weekend's 4-0 victory over West Ham United.
"The team was confident from the first minute," said Mourinho after a result that means United have scored four goals in their first two league games for the first time in 110 years.
"I like to see the confidence that the players are having in this moment. There's no panic. The team was always confident and always trying to move the ball. I'm really happy."
Sadio Mane struck in the 73rd minute to earn Liverpool a 1-0 home win over Crystal Palace and get Jurgen Klopp's side up and running after their 3-3 draw at Watford on the opening weekend.
Beaten 3-0 at home by Huddersfield Town in their first game, Palace produced a much more solid display, but were undone with 17 minutes to play when Mane seized upon a loose ball to fire home.
"It was a hard job. I have no idea how much we had the ball, but it was a lot," Klopp told Sky Sports.
"We were really awake and defended really well and I loved the goal because it was not a clear situation. We had other chances, but I am completely happy with how we did the job today."
Leicester, the 2016 champions, bounced back from defeat at Arsenal to beat Brighton and Hove Albion 2-0 at the King Power Stadium through Shinji Okazaki's first-minute goal and a Harry Maguire header.
Charlie Austin scored a 93rd-minute penalty to give Southampton a last-gasp 3-2 home win over West Ham.
Goals from Manolo Gabbidini and Dusan Tadic, the latter a penalty, put Saints 2-0 up and West Ham new boy Marko Arnautovic was sent off in between for an elbow on Jack Stephens.
Javier Hernandez's first two West Ham goals drew the 10 men level, only for Austin to secure victory from the spot after Maya Yoshida was fouled by Pablo Zabaleta.
Hal Robson-Kanu was hero and villain as West Bromwich Albion won 1-0 at Burnley.
The Wales striker came off the bench to score a 71st-minute winner, but was then sent off for leading with his arm in a challenge on Matthew Lowton.
Brazilian youngster Richarlison scored his first goal in English football and Etienne Capoue opened his account for the season as Watford claimed a deserved 2-0 win at Bournemouth.