Kubernetes Scheduling: Preference of nodes on one another

General Discussions
1

Asking for help? Comment out what you need so we can get more information to help you!

Cluster information:

Kubernetes version: 1.25
Cloud being used: GKE

I have a GKE cluster with multiple node-pools with multiple business use cases, i want to separate workloads onto particular node-pools. But if the specified node-pools reaches the specified maximum limit of nodes then it should go to another node-pool with GCE Spot instances. Below is my configuration.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: scheduling-test
  name: scheduling-test
spec:
 replicas: 1
 selector:
 matchLabels:
  app: scheduling-test
template:
metadata:
  labels:
    app: scheduling-test
spec:
  containers:
  - image: nginx:latest
    name: nginx
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        preference:
          matchExpressions:
          - key: bu
            operator: In
            values:
            - pool-1
  tolerations:
  - effect: NoSchedule
    key: bu
    operator: Equal
    value: pool-1

This is scheduling the pods on both the node-pools pool-1 and spot-pool, any inputs on how should i add scheduling predicates in this scenario?