Beginning work on the Lampcms API

My Question on Support site for LampCMS project

Yes, the Lampcms will have an API. It will be one of the few Open Source Q&A programs that has own API.

Before I beging working on the API, here are some basic outlines of what it will do:

1) Output will be in JSON or XML, but first I'll work on JSON version since it's much simpler. But right from the start the output will be handled by the formatter class. That class will accept raw data (php array or object) and convert it into JSON or XML. So first I'll just write JSON formatter)

2) It will be read/write API. First I'll work on read functions, then on write.

3) Good news and bad news about write API. Good news - it will be OAuth2, which is harder to write for me but easier to use for clients. Bad news - this will be one module which that will not be free. Yes, it will be the first premium module. After all, if you need the write API you are probably building a very interesting project with the purpose of making a profit. So it will not be a big deal to pay for this module. It will not be expensive anyway.

So, here you have it - a free read API with support to get latest questions, unanswered, by tag(s), support for "startwith" and "limit" and "sort by" options

Support to get all answers for a question with "startwith", "limit" and "sort" options and support for include/exclude comments from data

Support for questions for specific user, answers by specific user.

Maybe other options like get following/followers per user, but that will come later.

Any suggestions? Post them here.

Click here to post your reply

My answer to "Is there any way to include images in my post?"

This is my answer to a Question on Support site for LampCMS project

There are some posts with images on this site. I am not sure how they did it. There is no support to actually save an image to the server but there is a way to just include an img tag that points to image hosted somewhere else. 

My answer to "Is there any way to include images in my post?"

This is my answer to a Question on Support site for LampCMS project

There are some posts with images on this site. I am not sure how they did it. There is no support to actually save an image to the server but there is a way to just include an img tag that points to image hosted somewhere else. 

My answer to "Working on "Add YouTube" video"

This is my answer to a Question on Support site for LampCMS project

Start testing!
You can use this "Youtube" button now.
Just add some videos to your question or answer and see if it worked.
It still not the 100% polished, but it works with both inserting video and then playing in by replacing thumbnail with actual player.

My answer to "A test of Adding YouTube video"

This is my answer to a Question on Support site for LampCMS project

This is a reply with YouTube video in it.

My answer to "Some questions regarding lampcms"

This is my answer to a Question on Support site for LampCMS project

Lampcms Can handle such load if you do few things right:
1) Do NOT run php as mod_php under apache. Instead run it as fastcgi, and not only that, but also run it under control of php-fpm. php-fpm comes with php 5.3.3 now, so you should just set it up. Make sure you have anough ram on the server and allocate about 250 php processors to run. This will be more than enough because 1000 requests are not really at the same time - php will not take more than 1/4 second per page on Lampcms, some pages are rendered in under 50 millisends here! This is true, even though my server is just plain dual core, 2 hard drives....

You can still use Apache but do not use mod_php, configure it to run php as fastcgi.
Ideally you should use NGINX or Lighttpd server for php - they are capable of handling much higher load.

2) Always use APC. This is one of the easiest thing is the world to do. APC is not even a server - you don't need to start it up, all you have to do is make sure apc extension is enabled in php.ini (and of cause you must have apc extension installed)

3) For best results run MongoDB on separate server, but make sure it's really close to the web server, I mean on the same subnet. This is not necessary, you can perfectly run the MongoDB and web server together on one box, just make sure you have enough RAM. I recommend minimum of 16Gig if you run Mongo and Web server together.

4) Scaling will be done mostly by MongoDB - it supports "horizontal scaling", which basically means "Just add another server" and with minimal or even no downtime. MongoDB will take care of it, you need to know how to do this of cause. You can learn more on MongoDB site and on their discussion list.

You can also scale by adding extra web server. One of the easiest things to do is to have dedicated server for images, JavaScript and css files. LampCMS supports this and even has config options in !config.ini to define urls of Image and CSS servers.

For serving static files Apache is preferred choice is it has excellent in-memory cache module, which you should always use for static files.

If you need to add more web servers then there are ways to do that too. There are some common time-tested ways to do this like round robin DNS. Just install copies on Lampcms scripts on several servers, point them to the same MongoDB database or cluster of databases and you should be able to handle millions of visitors a day.


I recommend you start with just 2 servers: one web server and one database server. Put as much ram on your DB server and more importantly make sure you can add more RAM later. Start with 16Gigs of ram and have room to expand it like 4 times just in case you will even need it. 16Gigs should be enough for start.

Make sure hard disks are fast. RAID 10 is the best choice.

Web server does not need to be super fast, 2 hard drives - one for system files, one where the program is. If you like logging then it's best to log to dedicated disk, so 3rd disk for that.

If you sending out lots of emails then you may need another server as email server. You may also use some type of messaging system like ActiveMQ to send mail jobs to mail server. LampCMS has EventDispatcher, so you can just write simple php class that listens to events that need to send out mass emails then use messaging like ActiveMQ (php has libraries for it) and then just post messages to ActiveMQ server. All these will be done after the page renders, so these things do not add to page load at all. You need to use fastcgi under control on php-fpm for that. I already said this like 100 times on this forum - use php-fpm and you'll have Java-like performance on php.

My answer to "Thanks and some feature requests"

This is my answer to a Question on Support site for LampCMS project

Thanks.
This site already has the "Reputation" system just like on StackOverflow. You do earn points for votes and for w hen someone selects your answer.

You also earn points for posting your Q and A to Twitter, Facebook. Tubmlr, Blogger.

You also gain certain extra permissions for reaching certain score. This is all configurable. So the Points system is already fully implemeted.

As for badges - no. I am not sure I want this. I mean, what are we a boyscout club? I mean badges are great for when you in boyscout, but seriusly, I have never, ever paid any attention to anyone's badges (including my own) on StackOverflow. What this means to me is that there is a considerable work required to add a feature that I personally totally ignore on other sites.

Badges is just one of those features that I will do if someone sponsors it. I mean, I am open for coding sponsored features - you pay, I code. New code becomes available as feature - for everyone, not just you. Your name and link goes into "sponsors" category. A Win-Win.

As Far as "not as feature rich" as So and So... Take a look at what we already have. I think it has more features than "so and so". Does "so and so" have "Dot folders"? Does "so and so" have indicators of "Questions you follow"?

Does "so and so" have ability to follow tags, follow users and follow questions?

Does "so and so" have feature when words that match your tags are wrapped in <em> tag which can boost SEO for these words?

Does "so and so" have "Sticky thread" feature to keep certain posts on top of list all the time, like an announcement?

Does "so and so" have ability to post question or answer to Twitter, Facebook wall, Tumbl blog and Blogger blog - all at the same time?


Does "so and so" have excellent programming code editor/highter which is much better than one used on StackOverflow?


I am not sure about all other sites but at least one of the sites you mentioned does not have support for UTF-8 Tags - they only allow tags to be in English. On this site tags can be any word in any language - arabic, chinese, even html tags or percent sign can be in tag.

Does "so and so" have ajax based sort in tags and users pages?

Not as feature rich as StackOverflow? Think about it - SO does not even have their own user signup system - you MUST use open ID, there is no way to just create own account on SO. How lame is that?


Yes, this site does not have all the features that all other sites have. We don't have "Favorite" feature where you mark question as favorite - and it's saved to your favorites. But this is going to be added because I like it and I use it on stackoverflow, so it's going to be here too.

Yes, we don't have badges but I explained my position.

We don't have Internationalization yet but it's on a todo list and when I write it it's going to be better than "so and so" sites because I'm am planning to write complete locatization library not just translation. What I mean is not just language selection but locale like en_US, en_UK, etc... The locale also results in formatting date/time differently, formatting decimal points. currency etc.

My answer to "Code Editor is here"

This is my answer to a Question on Support site for LampCMS project

I think I fixed it. Testing - first html code, then php then css

<div class="someclass">My text</div>

now some php:

if(2>1 && $ok){
        run($ok);
} else {
        exit(1);
}

now css:

table {
        border-collapse: collapse;
        border-spacing: 0;
}

Code Editor is here

My Question on Support site for LampCMS project

OK, ready for this? I made some improvements to the RTEEditor. Resizing widget is gone but Link insertion is working much better and there is an Image insertion widget now.

I have also added a Code style highlighter to the editor. This is something I added to the YUI Editor myself. It is actually fairly complicated thing to add. It's a 3 part process: 1) adding the working drop down menu that would wrap the selected text in special tag that will later be used to format the code.
2) Some changes on the server to make sure it is parsed correctly by html parser
3) Adding couple of extra js files to page to do the actual code highlighting. I used an existing JS code highlighter code from http://www.dreamprojections.com/syntaxhighlighter/

Of cause I modified it to work the way I want it to.

And there is also a small step 4 - admin can enable this in !config.ini file by setting
ENABLE_CODE_EDITOR = true

It's false by default as for most non-programming sites this "Code style" tool will cause more confusion.


I think it beats code style editor/highlighter used on StackOverflow hands down. You can select the programming language, even CSS or HTML code as different programming codes are highlighted differently.

I'm sure StackOverflow is a great site, I use it all the time, that's where the crowd is. But they would be so much better if they use RTE from Lampcms instead of their Lame-o MarkDown editor.

Enjoy. And feel free to post some code just to try it out. It's OK.

My phpcode

if(2>1){
        run($nice);
} else {
        exit(1);
}

Click here to post your reply

Code Editor is here

My Question on Support site for LampCMS project

OK, ready for this? I made some improvements to the RTEEditor. Resizing widget is gone but Link insertion is working much better and there is an Image insertion widget now.

I have also added a Code style highlighter to the editor. This is something I added to the YUI Editor myself. It is actually fairly complicated thing to add. It's a 3 part process: 1) adding the working drop down menu that would wrap the selected text in special tag that will later be used to format the code.
2) Some changes on the server to make sure it is parsed correctly by html parser
3) Adding couple of extra js files to page to do the actual code highlighting. I used an existing JS code highlighter code from http://www.dreamprojections.com/syntaxhighlighter/

Of cause I modified it to work the way I want it to.

And there is also a small step 4 - admin can enable this in !config.ini file by setting
ENABLE_CODE_EDITOR = true

It's false by default as for most non-programming sites this "Code style" tool will cause more confusion.


I think it beats code style editor/highlighter than on StackOverflow hands down. You can select the programming language, even CSS or HTML code as different programming codes are highlighted differently.

I'm sure StackOverflow is a great site, I use it all the time, that's where the crowd is. But they would be so much better if they use RTE from Lampcms instead of their Lame-o MarkDown editor.

Enjoy. And feel free to post some code just to try it out. It's OK.

My phpcode

if(2>1){
        run($nice);
} else {
        exit(1);
}

Click here to post your reply

Should we use simpler Link and Image tool in Editor?

My Question on Support site for LampCMS project

I know that adding link in Editor brings up a form that is somewhat messed-up, some times more messed up than others. I found that the Editor resize utility, while is kind-of cool and I like it, cases the problem with the link form and also with image form (we don't have image tool right now but I tested it)

So the solution would be to either get rid of resize utility in Editor or use simple prompts to enter url for link and url for image.

The Editor will still be auto-resizable, meaning when you keep typing longer posts and need more height, the Editor will just auto-expand.

You can also see examples of simple Link and Image tools here on YUI demo page: http://developer.yahoo.com/yui/examples/editor/simple_adv_editor.html

I am leaning more towards getting rid of resize and keeping advanced link and image options, but this is still open for discussion.

Click here to post your reply

Added Blogger API support

My Question on Support site for LampCMS project

We now have an option to post Question or Answer to user's blog on Blogger via bloggerAPI.
Just like our Tumblr module it uses OAuth authentication.

Use can "connect" blogger blog from their profile page or from the "Ask" or "Answer" form when they click the "Post to blogger" check box and their account is not yet connected to Blogger.

More social features means more traffic for your site. Imagine a user with popular blog decides to share a question on a blog. That would mean a lot more new visitors to your Q&A site.

In the future we may also be able to monitor that post on blogger an any comments made on blogger will be imported as comments to that question or answer.

Also comments added to Question or Answer or an answer to a question made on the site may be posted to blogger's post as comments. This of cause is pretty dangerous feature because if someone posts a spam on your site we sure don't want to propagate that spam to someone else's blog.

Click here to post your reply

My answer to "Is there any way to include images in my post?"

This is my answer to a Question on Support site for LampCMS project

There are some posts with images on this site. I am not sure how they did it. There is no support to actually save an image to the server but there is a way to just include an img tag that points to image hosted somewhere else. 

Tumblr support added

My question on Support site for LampCMS project

I finished the TumblrAPI module.
User can now send Question or Answer to their Tumblr blog at the time of posting.

It uses OAuth authentication, just like Twitter API.

So source is on github.

I will work on few minor changes next like not including deleted items in views even for administrators
Instead there will be a separate controller to view deleted items and only administrator will be able to use it.

Click here to post your reply

Google Blogger OAuth API

After getting Oauth token and secret from Google for blogger
Check out this "playground" http://googlecodesamples.com/oauth_playground/index.php
it's so hard to even find the details of how to register your app with Google, what
url to use to request Oauth Token for blogger API.

When I find it again (takes a lot of wasted time to just find the correct url for documentation as Google scattered the blogger API documentation all over their sites - some in Oauth data related pages, some on Blogger API related pages...)

Some piece meal bits of data for Google OAuth API developers:

http://code.google.com/apis/gdata/articles/oauth.html

http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html

You will need to register your domain with Google. This included downloading some lame html file from Google then uploading it to root dir of your site and then clicking "verify" button, basically proving to Google that you are the admin of that domain.

Once that is done you need to go here https://www.google.com/accounts/UpdateDomain

and enter some string that you must create yourself - some type of long secret string, into the

"Target URL path prefix"

That will become your app's OAuth Consumer Key and Google will generate the OAuth Consumer Secret value for you. This is the lamest and most unintuitive way I"ve seen in any APIs to get your OAuth Consumer API key and OAuth consumer secret. But that's how it is with Google, it's because they don't really care about the developers(about schmucks like you)

Now you will probably be able to use OAuth protocol, but of cause nothing is "easy" when it comes to Google's implementation of OAuth - remember this kids - Google does not care about developers and does not want to provide clear and easy-to-understand or even easy-to-find documentation.

Some details are here

http://code.google.com/apis/blogger/docs/2.0/developers_guide_protocol.html

can use this url to get list of user's blogs: https://www.blogger.com/feeds/default/blogs

It will be in the superbly fucked-up format written by incompetent fools at Google.
Even though it's in XML format, there is no entry for "blogID" and you must extract it from the tag using regex.

The url for the POST - to post your entry to blogger using api is also not easy to extract. It's buried in the

Why the fuck the type="html"? This is an XML file - supposed to be pure data, why html?
That's not even the worst part.

To extra the url for post you need to find an entry in the "summary" tag that looks like this:


There is no easy way to extract this entry because there are a bunch of other very similar entries. Your only clue to use is that the "rel" attribute must end with #post How do you extract this entry? Is there a way to use XPATH when the only condition you have is that value of an attribute ends with #post?
Maybe if you can be certain that value will be "http://schemas.google.com/g/2005#post" then yes, it's possible to find this entry easily and then use the value of href

But it's probably easier to just extract and store the blog-id by extracting it out of the tag.