Myujth

I have a paginated response from an URL, I want to keep on hitting the next page URL which I get from the previous response and keep on collecting items till I don't have a "nextPage" URL in my response. How to achieve this in a reactive way using spring boot WebClient from WebFlux with out blocking?

Request1: 

 GET /items
 response: 
 
 items: [...]
 nextPage: "/items?page=2"
 


 Request2: 

 GET /items?page=2
 response: 
 
 items: [...]
 nextPage: "/items?page=3"
 


 Request3: 

 GET /items?page=3
 response: 
 
 items: [...]
 nextPage: null
 

Here I have created mock urls
https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items
https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items?page=2
https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items?page=3

How can I extract all Items from the above responses in a reactive way without blocking?

Using expand, this can be achieved.
Based on the mock urls provided by you.

public Mono<List<Item>> getItems() 
 String url = "https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items";

 return fetchItems(url).expand(response -> 
 if (response.getNextPage() == null) 
 return Mono.empty();
 
 return fetchItems(response.getNextPage());
 ).flatMap(response -> Flux.fromIterable(response.getItems())).collectList();


private Mono<Response> fetchItems(String url) 

 return client.get().uri(url).retrieve()
 .bodyToMono(Response.class);
 

You can achive a desired effect using exapnd:

@Test
public void usingExpand()

 Request innerData = new Request(null);
 Request middleData = new Request(innerData);
 Request rootData = new Request(middleData);

 Mono.just(rootData)
 .expand( t -> Mono.justOrEmpty(t.netxPage))
 .flatMap( t -> Flux.fromIterable(t.items))
 .subscribe(System.out::println);



public static class Request 
 List<String> items = new ArrayList<>();
 Request netxPage;

 public Request(Request netxPage) 
 this.items.add(UUID.randomUUID().toString());
 this.items.add(UUID.randomUUID().toString());
 this.netxPage = netxPage;
 

The above code shuld produce the following result:

dc78317c-5552-4723-90db-5392c67655be
32ff12bb-5be1-415e-b481-dab85d9157dd
cf1e3f36-a8e2-414d-90a2-7708eeedc5be
91a6bc14-a396-483d-a66a-80bb98dc1968
c95adae3-8e6f-489b-8a9d-4cea3080e150
d6f8fe01-2c50-4574-958c-ec675331bb25

Two UUID from each data object.