Option to omit HTML from collated JSON #48
Comments
|
I can try implementing that if CC @simonihmig. |
|
@lolmaus did you try setting the content-types configuration? 🤔 https://github.com/empress/broccoli-static-site-json#content-types |
|
@mansona I did not. I assume removing I only want to remove the HTML from the collated response for the I believe this should be a rather common requirement. |
|
@lolmaus I don't think that this would be a common requirement 😉 what you could do is that you could have 2 instances of broccoli-static-site-json (one with contentTypes and one without) and merge them with broccoli-merge-trees with |
Loading the content of all posts on the posts index page makes no sense to me because I'm not showing full content on the index. With a large library of blog posts, it will also affect performance on entry-level mobile phones, smartwatches, etc. Why is it common to choose wasting user's time, traffic, CPU and memory? Well, there is one legit case: to speed up page transitions for users with powerful devices and fast internet. But I doubt that it is entirely uncommon to care for other users. Note that there are cases when you want to load all posts' metadata on individual post pages. For example, you want a table of contents, or prev/next links labeled as The whole point of loading posts over network is to load only the currently required one for each page, rather than all of them.
That works, but it is not convenient. It will require messing with adapters to abstract away the fact that different names are used for different types of requests of the same resource. Erm, that's not a legit argument because you need to modify the adapters anyway. But note that there is no option to opt out of generating individual files, resulting in two sets of them generated! It will also double the amount of Why are you opposed to adding an option like What is the downside of having this option in |
|
excuse me @lolmaus but I don't think your negative response here is warranted. The scenario that I was saying was not a common requirement is having a difference between an aggregated view of the data and the individual view of the data. I will do my best to answer your query as it is written, but I will ask that you follow the Ember Community Guidelines when continuing this conversation, especially the section about Be empathetic and respectful. Firstly I have a question for you: why are you continually asking about "posts" when you're discussing this? Are you using this library directly or are you actually using empress-blog (which is one of the main users of this library). The reason why I'm asking this is because
There is a rudimentary pagination implementation that might interest you: https://github.com/empress/broccoli-static-site-json#paginate
This is some of the main reasons why I think what you're trying to do is not a common case. There is no way to solve this in a generic way that will solve everyone's use case if there is every any differences between the aggregated view and the individual view. Again I will re-iterate that this is a low level library for file conversions, this should be thought of as building a static API that is valid JSON:API rather than "integrating specifically with ember" (even though I give a helpful example of how an Ember dev might use this in the markdown)
Well this is why I suggested using MergeTrees with
how much broccoli-static-site-json configuration code do you have? In most cases (unless you're doing something relatively complex like empress-blog does) you should have pretty much a single line for each tree of files no? (maybe split over multiple lines for attributes etc.) but logically speaking you should have a single statement for your broccoli-static-site-json tree no?
I never said I was opposed to anything 😄 I was merely trying to challenge an assumption that you had in your assessment that something would be a common requirement. Although it's worth noting that no option to any library is ever free, if we add this we need to test it, maintain it, upgrade it if we restructure this library (again), make sure the documentation is correct, make sure the documentation is updated if there are any reasonable changes to the implementation, and so on. And this is not even counting the situation where it would add implicit complexity to a library that is already arguably overly complex for the simple feature-set that it is trying to achieve. Not only that, we have seen that the library can be effective in very large implementations without hitting any of the major issues that you are suggesting are inherent as a result of the architecture of the lib 🤔 |
|
Hey @mansona, thank you for a detailed response!
Please accept my apology; I assure you that offence was not my intention. 😔 It is my fault that my excitement and passion translates to arrogance in writing.
I use it directly directly to add a blog section to a Prember-driven app, in a very similar way to
Well, we've discussed it many times over already: Empress is way too biased. It's not like any other Ember addon, it's more like a prebuilt app which happens to be bundled as an addon. I need flexibility and
It will only make things worse:
Every API is designed for a specific purpose. Some APIs are designed to be used publicly, but I doubt that a static blog is one of them. And even it were, I still don't see a problem: I'm not asking to deprive all But note that most APIs are designed for a specific frontend, in this case a blog website. Almost all blog websites are consumed with a browser, not an API client (well, except for RSS, but it's generated separately). And if my particular website does not have a case for loading full content in As for "rather than integrating specifically with ember", I don't believe Broccoli has any mention-worthy usage outside the Ember community. It has only 88 dependants on npm, with
OK, that works. I assumed it would not work because file names would be different and will not overwrite each other. But I now see that I can use identical names and only change the
Well, yes, it's a single statement, but:
Also, using
And I ended up violating community guidelines. ☹ I'm sorry for that, of course I know you're acting in good faith and never did/said anything that would allow my arrogance. I just hope you see that I am acting in good faith too. I would like to contribute to a wonderful tool that you've built, fix a pain point I'm having (and I still believe it should be a common practice for static blogs, and many
If your argumentation is taken with full seriousness, it would seem that no library is worth writing. I believe it should be a matter of comparing the maintainers' costs to users' costs. Most of what you mentioned is a one-time thing: after the feature is implemented, all the documentation updated and all the issues resolved, there is little to no effort required to maintain it. But if it's not implemented, users will have to be paing the cost indefinitely: every time someone cares about the payload size, they need to go learning what broccoli trees are and how they work. Provided they come up with this idea in the first place. OK, I'm not wasting any more of your time. |
|
@mansona Thank you for your const StaticSiteJson = require('broccoli-static-site-json');
const broccoliMergeTrees = require('broccoli-merge-trees');
const blogPostsInputFolder = 'content/posts'; // input folder
const blogPostsConfig = {
attributes: ['title', 'snippet', 'description', 'date', 'author', 'layout', 'twitter'],
contentFolder: 'api/posts', // output folder
type: 'blog-post', // JSONAPI resource name
};
const blogPostsJsonFrontmatterOnly = new StaticSiteJson(blogPostsInputFolder, {
...blogPostsConfig,
collate: true,
contentTypes: [],
});
const blogPostsJsonFull = new StaticSiteJson(blogPostsInputFolder, {
...blogPostsConfig,
contentTypes: ['html'],
});
module.exports = {
name: 'content',
treeForPublic() {
return broccoliMergeTrees([blogPostsJsonFrontmatterOnly, blogPostsJsonFull], {
overwrite: true,
});
},
};With my suggestion implemented, it would look like this: const StaticSiteJson = require('broccoli-static-site-json');
const blogPostsJson = new StaticSiteJson(
'content/posts', // input folder
{
attributes: ['title', 'snippet', 'description', 'date', 'author', 'layout', 'twitter'],
collate: true,
contentFolder: 'api/posts', // output folder
contentTypes: ['html'],
collatedContentTypes: [],
type: 'blog-post', // JSONAPI resource name
}
);
module.exports = {
name: 'content',
treeForPublic() {
return blogPostsJson;
},
}; |
Thanks for saying that, it makes a big difference. I appreciate that emotions can run high, especially when you are trying to achieve something and a library isn't providing what you need it to do. Now let's get onto the rest of your questions.
Yes you're right here, empress-blog is intended to be biased and brought it up as a reference rather than a suggestion for you to use it. Sorry If I have forgotten that we have discussed empree-blog already 🙈 I am starting to get to the point where I am forgetting who I've discussed it with, maybe this is some sort of measure of success? 😂
So we have actually solved this ToC problem for Guidemaker and Field Guide in different ways, but both of them essentially use a custom broccoli-tree to build a separate file that can be requested at run time. I'm working on a more generic solution for all Empress projects but for now ToC building is quite a hard problem unfortunately 😫
again with the "static blog" as an example 😂 while empress-blog is the most "popular" project to use this library I doubt that the aggregate of page views on all empress-blogs would match its usage on the Ember Website. The easiest example of this being used right now is the fact that the team page is build using this sort of "static API" so you can build a quick ember app that directly requests https://emberjs.com/data/team-members/all.json or https://emberjs.com/data/team-members/chris-manson.json as necessary 😉 this may seem like a toy example but there is a plan to integrate this into a new quick-start tutorial at some point in the future. The reason I bring up these examples is that I really want to hammer home the fact that you can't think of this library is "able to build something for your frontend" but instead it should be thought of as "building a valid static API"
While I would be very happy if someone did use broccoli-static-site-json outside of an Ember context, my first emotion would be an extreme surprise 😂 by suggesting that you think of this as a separate thing to "something that integrates with Ember" is not saying that it should be flexible enough to work outside Ember, but instead I'm trying to re-enforce the point that when working with broccoli-static-site-json you need to have a different mental context (as I have said before). Think of it as building an API, not getting data from Markdown into an Ember app 👍
If keeping each pair in sync is an issue I recommend wrapping this functionality in a little function that returns the merged trees.
I'm sympathetic about this issue because I have believed for a long time that the broccoli docs could vastly be improved. I know broccoli dev is currently an expert-level task in the Ember community but that doesn't really need to be the case. I think that with a little hand-holding and a few tutorials this issue could be solved without too much effort. I also loved pretty much everything that @oligriffiths was doing in this area a little while ago 👍
So you're hitting against 2 things here. Yes, the whole "string defaults to a folder tree" thing is a bit strange with broccoli, but the API that you're talking about is a bad decision on my part when I first created broccoli-static-site-json 😞 it is unnecessarily confusing and has caused quite a few issues over the years. I guess I'm only human 🤷 🙃 😂
I agree, at least until I discovered https://github.com/stefanpenner/broccoli-stew and the
while this is technically true, merge-trees is used so much in everything that uses broccoli that it's practically fundamental and adding that dependency will likely not actually add a new entry to your package-lock. It's actually so fundamental that we could probably propose adding it to the broccoli lib by default 😂
that's a bit of a leap I think 😂 saying something is never free doesn't mean that it automatically costs Infinity and therefore would not be worth writing
Well, this is already not true. The "little mistake" that I described earlier in this comment already cost me an @billybonks quite a lot of time in his effort to modernise this repo by properly using broccoli trees in #32 It's quite a well-documented fact that the majority of the cost of software is in the maintenance, I did a quick search there and found plenty of sources (from the last 20 years) estimating that it's somewhere between 60% and 75%. That doesn't mean that we're going to spend hours working on this one feature if we add it, it's my opinion that adding complexity like this to an already complex library will likely slow us down in the future because we need to keep the implementation working while we refactor or change the library to be more efficient. And that's not even taking into consideration the fact that if we add this config option I believe that it would allow (or actually encourage) people to create invalid JSON:API static endpoints.
I mentioned in my previous post that we have an alternative idea on how to improve the payload size of the collated document, but I'm very convinced that this is an application concern and not something that the low-level library should be doing.
I don't see any of this as a waste of time 🎉 one thing that I know I'm lacking is actually writing down the architectural goals on libraries like this one, being able to respond to questions like this really helps to implicitly document the motivation behind this stuff so I appreciate you sticking with it and |
|
And as for your code example, I would probably have done something closer to: const StaticSiteJson = require('broccoli-static-site-json');
const broccoliMergeTrees = require('broccoli-merge-trees');
function buildJsonTree(inputFolder, config) {
let contentFolder = `api/${inputFolder}`
const blogPostsJsonFrontmatterOnly = new StaticSiteJson(`content/${inputFolder}`, {
...blogPostsConfig,
contentFolder,
collate: true,
contentTypes: [],
});
const blogPostsJsonFull = new StaticSiteJson(`content/${inputFolder}`, {
...blogPostsConfig,
contentFolder,
contentTypes: ['html'],
});
return broccoliMergeTrees([blogPostsJsonFrontmatterOnly, blogPostsJsonFull], {
overwrite: true,
});
}
module.exports = {
name: 'content',
treeForPublic() {
return buildJsonTree('posts', {
attributes: ['title', 'snippet', 'description', 'date', 'author', 'layout', 'twitter'],
type: 'blog-post', // JSONAPI resource name
})
},
};that way the function essentially encapsulates the thing that you're trying to achieve here logically 👍 |
Hi! Using
broccoli-static-site-jsonwith Prember.I would like to show a list of blog posts on the blog index page. But loading the content of all the blog posts seems excessive. I would like to load only the metadata (frontmatter).
It would be nice to have an option to enable this.
The text was updated successfully, but these errors were encountered: