Position rack in center of oven and preheat to 350°F. Spray 9-inch-diameter cake pan with 2-inch-high sides with nonstick spray. Line bottom of pan with parchment paper round.
Finely grind almonds in a food processor. Transfer to medium bowl; whisk in flour, baking powder, and salt. Using electric mixer, beat butter in large bowl until fluffy. Add 1 cup sugar; beat until well blended. Add eggs 1 at a time, beating well after each addition. Beat in vanilla and almond extract, then flour mixture just until incorporated.
Transfer batter to prepared pan; spread evenly and smooth top with spatula. Gently press plum slices, flesh side down, into batter in spoke pattern around outer rim and center of cake, placing close together. Mix cinnamon and 4 teaspoons sugar in small bowl. Sprinkle over plums.
Bake cake until tester inserted into center comes out clean, about 50 minutes. Cool cake in pan on rack 20 minutes. Run small knife between cake and pan sides to loosen. Invert cake onto platter; remove parchment paper. Place another platter atop cake. Using both hands, hold both platters firmly together and invert cake, plum side up. Cool cake completely. Cut into wedges.